class: center, middle, inverse, title-slide # Applied Data Analysis for Public Policy Studies ## Multiple Regression Model ### Michele Fioretti ### SciencesPo Paris </br> 2022-08-29 --- layout: true <div class="my-footer"><img src="../img/logo/ScPo-shield.png" style="height: 60px;"/></div> --- # Recap from last week * Causality versus correlation * The Potential Outcome Framework a.k.a. Rubin's Causal Model * Randomized controlled trials (RCTs) -- ## Today - Multiple Regression Model * **Multiple Regression Model** * *Extensions:* standardized regression; log models; interacting variables * Empirical applications: * *Class size* and *student performance* * *Wage*, *education* and *gender* --- # Class size and student performance * Let's go back to Angrist and Lavy's (1999)'s analysis of the effect of class size on student performance in Israel. -- * With a **simple linear regression**, we found that class size was positively ***associated*** with students' scores in maths and reading. --- # Class size and student performance: Raw relationship <img src="chapter4_files/figure-html/unnamed-chunk-1-1.svg" style="display: block; margin: auto;" /> --- # Class size and student performance: Raw relationship <img src="chapter4_files/figure-html/unnamed-chunk-2-1.svg" style="display: block; margin: auto;" /> --- # Class size and student performance: Raw relationship .pull-left[ <img src="chapter4_files/figure-html/unnamed-chunk-3-1.svg" style="display: block; margin: auto;" /> ] .pull-right[ ```r lm(avgmath ~ classize, grades) ``` ``` ## ## Call: ## lm(formula = avgmath ~ classize, data = grades) ## ## Coefficients: ## (Intercept) classize ## 57.7939 0.3175 ``` * *Interpretation:* ***On average***, a 1-student increase in class size is ***associated*** to a 0.32 points increase in average math score. ] --- # Class size and student performance * Let's go back to Angrist and Lavy's (1999)'s analysis of the effect of class size on student performance in Israel. * With a **simple linear regression**, we found that class size was positively ***associated*** with students' scores in maths and reading. * This is intuitively unexpected, and contrasts with the simple results from the *STAR* randomized experiment. -- * Could it be that some other variable may be related to class size ***as well as*** students' performance? * In particular, we mentioned the **location effect**: large classes may be more common in wealthier and bigger cities, while small classes may be more likely in poorer rural areas. -- * Let's investigate this hypothesis. --- # Class size and student performance: Confounders .pull-left[ Link between **class size** and **the share of student who come from `disadvantaged` background** in the class. <img src="chapter4_files/figure-html/unnamed-chunk-5-1.svg" style="display: block; margin: auto;" /> 👉 On average, there is a greater % of disadvantaged students in smaller classes. ] -- .pull-right[ Link between **average math score** and **the share of student who come from `disadvantaged` background** in the class. <img src="chapter4_files/figure-html/unnamed-chunk-6-1.svg" style="display: block; margin: auto;" /> 👉 On average, the greater the % of students coming from a disadvantaged background, the lower the average math score. ] --- # Class size and student performance: Multiple regression * Suppose we want to know the effect of class size on average math scores, ***controlling for*** the fact that there is a negative relationship between the % of disadvantaged students and class size **AND** average math score. -- * To do so, we have to include both `classize` and `disadvantaged` variables as *regressors* in the regression. -- * As such we can obtain an estimate of the effect of class size on average math score, ** *purged* of the effect of the disadvantaged variable**. -- * The model we want to estimate becomes: $$ \textrm{mathscore}_i = b_0 + b_1 \textrm{classize}_i + b_2 \textrm{disadvantaged}_i + e_i $$ -- * This is ***multiple regression***! We will estimate this model in a few slides. Let's formalize what we have seen so far. --- # Multiple Regression's Purpose * Recall from two weeks ago, the **Simple Linear Model** can be written as `$$y_i = b_0 + b_1 x_i + e_i,$$` where `\(y_i\)` is the ***dependent variable*** and `\(x_i\)` is the ***independent variable***. -- ⚠️ Unless all other factors affecting `\(y_i\)` are uncorrelated with `\(x_i\)`, `\(b_1\)` **cannot be interpreted as a causal effect**. -- We need to **enrich the model** and take into account factors that are simultaneously related to `\(y_i\)` **and** `\(x_i\)`. --- # Mutiple Regression Model The expanded model can be written as: `$$y_i = b_0 + b_1 x_{1,i} + b_2 x_{2,i} + b_3 x_{3,i} + \dots + b_k x_{k,i} + e_i,$$` where `\(x_1\)`, `\(x_2\)`, ..., `\(x_k\)` are `\(k\)` regressors, and `\(b_1\)`, `\(b_2\)`, ..., `\(b_k\)` are the associated `\(k\)` coefficients. -- ***Estimation***: We obtain the values for `\((b_0, b_1, b_2, ..., b_k)\)` in the same way as before, using **OLS**. * `\((b_0^{OLS}, b_1^{OLS}, b_2^{OLS}, ..., b_k^{OLS})\)` are the values that minimize the **Sum of Squared Residuals**. * That is they minimize $$ `\begin{align} \sum_{i}{e_i^2} &= \sum_{i}{(y_i - \hat{y_i})^2} \\ &= \sum_{i}{[y_i - (b_0 + b_1 x_{1,i} + b_2 x_{2,i} + b_3 x_{3,i} + \dots + b_k x_{k,i})]^2} \end{align}` $$ --- # Multiple Regression Model: Interpretation For now assume both the dependent variable `\((y_i)\)` and the independent variables `\((x_k)\)` are numeric. > Intercept `\((b_0)\)`: **The predicted value of `\(y\)` `\((\widehat{y})\)` if all the regressors `\((x_1, x_2, x_3,...)\)` are equal to 0.** -- > Slope `\((b_k)\)`: **The predicted change, on average, in the value of `\(y\)` *associated* to a one-unit increase in `\(x_k\)`...** <br/> > `\(\quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad\)` **... keeping all the other regressors constant!** -- * Notice that the *keeping all the other regressors constant* is the only part that changes compared to SLM. * In other words, you are considering the indidivudal effect of the variable `\(x_k\)` on `\(y\)` **in isolation** of the effect that the other regressors might have on `\(y\)`. -- * **Link with causal inference**: Only the regressors included in the model are held constant, those that are not in the model can still vary and "bias" your estimates. --- # Multiple Regression with `R` * Very similar to simple linear regression: ```r lm(formula = dependent variable ~ independent variable 1 + independent variable 2 + ..., data = data.frame containing the data) ``` -- ## Class size and student performance: Multiple regression Let's estimate the model from earlier on by OLS: `\(\textrm{mathscore}_i = b_0 + b_1 \textrm{classize}_i + b_2 \textrm{disadvantaged}_i + e_i\)` ```r lm(avgmath ~ classize + disadvantaged, grades) ``` ``` ## ## Call: ## lm(formula = avgmath ~ classize + disadvantaged, data = grades) ## ## Coefficients: ## (Intercept) classize disadvantaged ## 69.94438 0.07168 -0.33958 ``` --- # Class size and student performance: Multiple regression ``` ## ## Call: ## lm(formula = avgmath ~ classize + disadvantaged, data = grades) ## ## Coefficients: ## (Intercept) classize disadvantaged ## 69.94438 0.07168 -0.33958 ``` ***Questions*** 1. How do you interpret each of these coefficients? 1. How do you explain the change in the `classize` coefficient compared to the SLM case? --- # Class size and student performance: Multiple regression ``` ## ## Call: ## lm(formula = avgmath ~ classize + disadvantaged, data = grades) ## ## Coefficients: ## (Intercept) classize disadvantaged ## 69.94438 0.07168 -0.33958 ``` ***Answers*** 1\. How do you interpret each of these coefficients? * `\(b_0\)` = 69.9: When `class size` and `disadvantaged` are set to 0, the *predicted* value of the average math score is 69.9. -- * `\(b_1\)` = 0.07: Keeping the share of *disadvantaged students* constant in the class, a 1-student increase in class size is ***associated, on average,*** with a 0.07 point increase in average math score. -- * `\(b_2\)` = - 0.34: Keeping the *class size* constant, a 1-percentage point increase in the share of *disadvantaged students* is ***associated, on average,*** with a 0.34 point decrease in average math score. --- # Class size and student performance: Multiple regression ``` ## ## Call: ## lm(formula = avgmath ~ classize + disadvantaged, data = grades) ## ## Coefficients: ## (Intercept) classize disadvantaged ## 69.94438 0.07168 -0.33958 ``` ***Answers*** 2\. How do you explain the change in the `classize` coefficient compared to the SLM case? * `\(b_1\)` decreases when the `disadvantaged` variable is taken into account. This was expected since part of the positive effect of class size was partly due to the smaller share of disadvantaged students in bigger classes. --- # Multiple Regression in 3D <div id="htmlwidget-ed8da50141c3e506ab09" style="width:5px;height:5px;" class="plotly html-widget"></div> <script type="application/json" data-for="htmlwidget-ed8da50141c3e506ab09">{"x":{"visdat":{"46a4bd5a405":["function () ","plotlyVisDat"]},"cur_data":"46a4bd5a405","attrs":{"46a4bd5a405":{"tickfont":{"size":7},"alpha_stroke":1,"sizes":[10,100],"spans":[1,20],"x":{},"y":{},"z":{},"type":"scatter3d","mode":"markers","name":"1 dot = 1 class","opacity":0.6,"marker":{"color":"blue","size":1.5,"hoverinfo":"skip"},"inherit":true},"46a4bd5a405.1":{"z":{},"type":"surface","x":[5,44],"y":[0,76],"name":"3D regression plane","opacity":0.7,"cauto":false,"colorscale":[[0,1],["red","yellow"]],"inherit":false}},"layout":{"width":5,"height":5,"margin":{"b":40,"l":60,"t":25,"r":10},"xaxis":{"tickfont":{"size":7}},"yaxis":{"tickfont":{"size":7}},"scene":{"xaxis":{"title":"classize"},"yaxis":{"title":"disadvantaged"},"zaxis":{"title":"avgmath"}},"hovermode":"closest","showlegend":false,"legend":{"yanchor":"top","y":0.5}},"source":"A","config":{"showSendToCloud":false},"data":[{"tickfont":{"size":7},"x":[28,26,22,15,32,34,34,30,26,31,32,32,31,39,38,31,33,33,30,28,34,29,34,34,36,23,19,32,31,29,28,28,17,38,38,38,36,35,25,29,26,16,30,40,20,22,29,21,20,30,30,29,29,12,24,26,32,26,33,30,31,30,27,28,27,29,24,30,35,35,36,22,25,29,22,31,26,35,32,31,29,28,32,33,32,27,26,27,34,33,34,22,20,34,34,26,24,18,22,21,24,43,24,33,41,13,32,32,23,23,39,40,41,40,37,39,40,25,30,32,27,27,27,28,35,30,34,23,20,25,27,23,33,32,36,32,33,31,35,36,32,31,32,22,36,34,34,34,38,26,25,24,25,33,34,35,39,28,25,34,34,34,34,40,32,36,35,29,35,38,39,38,39,20,38,36,36,10,30,21,23,25,19,21,29,15,30,28,22,31,32,18,33,24,21,26,25,26,9,27,22,20,33,26,26,24,27,23,26,23,22,27,26,28,26,28,24,25,26,25,27,14,21,24,30,29,34,36,33,36,35,35,31,30,34,21,21,25,29,29,27,31,24,24,35,33,32,21,22,20,21,41,39,22,23,24,27,26,33,32,18,22,30,31,33,20,15,29,30,37,37,20,17,20,21,25,23,28,28,23,29,30,28,25,16,25,26,27,29,26,24,24,39,38,37,22,20,31,30,31,40,40,40,24,27,27,40,26,24,21,22,23,34,22,20,12,27,21,34,32,32,37,34,25,29,23,23,19,19,32,32,40,40,39,36,36,29,29,28,21,27,20,21,24,14,13,27,29,27,36,39,35,27,33,38,37,33,33,22,28,28,22,34,21,20,25,27,32,31,31,12,25,24,20,22,21,20,32,26,28,28,23,21,23,29,30,26,24,17,27,21,27,27,24,25,25,40,40,26,26,36,34,23,22,27,11,22,33,33,34,25,23,23,22,40,40,37,37,31,26,38,40,30,30,40,40,40,30,28,21,22,29,29,19,21,22,39,33,33,31,31,38,28,35,38,32,34,33,34,32,34,33,32,34,34,39,40,25,25,24,23,39,39,35,35,30,28,23,29,22,22,18,19,33,30,32,20,31,31,40,41,40,36,37,36,36,27,32,35,35,37,23,25,17,23,22,20,24,26,14,35,34,40,40,40,38,33,32,30,39,38,38,38,40,35,37,32,35,34,35,27,20,12,27,32,28,29,28,37,34,33,36,29,29,34,37,35,38,21,20,26,22,24,24,24,26,37,26,37,34,26,26,39,37,20,20,32,22,33,31,23,9,33,33,26,26,29,26,28,22,16,22,40,40,38,24,9,27,34,30,35,35,40,22,22,21,22,19,31,32,21,38,37,20,17,33,16,24,20,33,33,35,21,22,20,20,40,14,29,29,32,23,23,30,33,20,36,34,32,35,36,32,34,29,29,29,26,25,28,32,28,10,36,33,30,29,30,21,30,33,32,28,30,34,34,35,25,20,14,11,34,34,35,35,30,28,31,29,30,33,38,38,40,38,29,27,29,25,38,30,33,34,34,35,34,19,19,38,38,36,21,21,22,12,31,14,28,27,21,29,30,33,33,38,39,39,36,38,38,40,39,37,37,37,26,24,21,30,30,37,36,35,24,15,38,38,37,24,21,27,23,35,37,38,21,20,15,24,24,23,22,29,11,29,30,27,35,33,31,35,24,27,20,23,25,27,30,29,30,19,34,34,31,33,29,21,20,20,22,32,32,31,21,31,34,31,19,22,20,16,29,32,31,35,38,34,28,31,32,35,22,23,36,37,21,20,34,37,38,20,32,34,38,36,35,36,37,25,34,27,29,28,17,14,31,25,37,39,11,36,39,35,38,40,37,34,32,31,42,43,39,39,28,25,23,23,23,20,21,22,21,38,25,30,31,31,23,21,33,35,28,11,27,27,34,28,23,32,38,34,36,25,29,19,30,28,41,41,28,28,24,23,24,28,30,24,27,39,39,40,39,31,32,40,40,40,40,38,40,40,31,28,34,36,26,12,32,37,34,36,22,24,27,31,30,28,22,24,29,30,33,30,32,32,38,10,17,33,31,39,39,38,32,33,34,34,33,28,30,26,31,33,15,33,33,31,34,33,33,28,35,37,36,38,39,27,28,29,28,30,26,27,28,28,34,27,35,27,24,24,25,32,29,22,21,25,29,32,24,25,28,27,25,22,34,33,31,27,27,37,40,36,37,38,36,35,37,20,24,22,30,28,31,28,31,31,34,33,25,26,37,37,35,38,37,37,39,40,40,34,33,28,26,28,40,40,32,33,31,21,23,26,28,38,40,35,35,37,36,36,32,31,29,32,36,19,34,32,35,23,19,19,34,29,32,31,32,33,30,34,36,37,36,37,37,37,37,37,37,14,25,25,29,25,34,34,32,31,26,24,23,40,40,40,40,31,32,31,21,19,31,26,27,20,23,23,35,33,37,37,27,26,29,29,29,27,26,27,28,33,32,34,30,36,38,35,41,40,21,26,27,23,38,37,36,40,39,35,30,33,33,38,39,36,38,40,40,31,32,33,35,35,34,34,34,30,25,37,35,35,36,36,37,36,37,36,31,33,29,34,37,36,31,30,34,30,39,33,33,40,41,40,35,36,36,34,36,32,40,40,40,40,36,39,35,36,38,39,26,30,30,30,31,35,30,28,30,31,33,36,37,39,21,37,36,40,36,39,38,37,36,38,34,35,29,32,33,31,34,29,40,39,30,34,35,34,35,23,34,37,33,28,27,35,36,35,31,27,31,24,27,32,33,31,33,36,37,39,39,40,37,31,36,32,39,41,40,31,30,40,40,38,38,34,33,32,33,32,34,31,35,35,35,34,33,36,32,33,32,41,29,32,34,26,27,30,38,36,34,36,36,20,22,22,31,32,30,33,33,31,30,30,30,34,16,20,37,37,5,17,26,26,28,24,28,24,21,16,35,35,22,25,30,15,22,38,37,36,34,26,23,36,37,38,38,39,38,26,25,26,27,29,28,40,40,31,31,26,27,22,25,33,35,22,27,35,35,38,39,37,33,28,31,32,32,34,32,33,30,28,32,32,32,28,33,24,27,20,20,31,34,30,35,34,34,31,31,25,25,40,41,14,21,36,36,24,23,37,40,38,30,35,33,34,38,38,36,31,28,28,38,37,28,26,26,27,39,38,25,35,37,36,32,37,35,35,33,25,22,28,25,39,38,24,29,27,37,37,37,37,39,23,21,28,26,31,10,31,36,31,31,33,39,40,34,35,36,37,33,33,29,22,39,39,18,25,25,22,25,26,24,26,22,32,29,21,26,26,25,26,32,31,30,28,26,31,20,22,22,21,19,27,28,31,27,33,32,34,32,27,23,26,27,26,40,35,32,29,28,27,25,28,22,30,29,28,29,29,19,20,8,29,24,32,33,17,28,23,24,25,27,29,35,34,12,27,23,35,35,21,38,40,40,40,39,30,39,37,29,30,31,33,41,41,40,28,24,33,33,22,21,32,38,28,30,24,26,26,29,29,25,25,28,28,32,26,23,37,34,30,29,28,33,33,33,20,21,40,33,33,33,31,31,26,21,29,26,30,31,25,33,30,35,30,29,35,36,38,38,33,29,34,28,12,29,23,22,26,29,25,27,32,32,30,28,24,27,36,36,36,27,27,29,30,25,41,40,38,40,40,22,24,37,37,30,29,33,28,31,39,39,40,22,23,41,32,31,30,38,38,38,37,39,35,36,36,36,34,38,39,38,42,36,36,39,41,39,34,23,24,20,29,28,28,33,33,33,38,39,40,37,39,39,40,34,36,34,35,35,27,30,35,33,37,37,31,32,34,34,12,32,34,23,27,26,37,37,38,30,34,37,35,28,37,28,27,29,26,18,38,24,36,24,41,40,29,32,18,18,30,31,34,32,30,31,30,30,34,31,12,32,29,32,35,28,27,33,32,28,28,35,36,28,29,38,28,30,28,33,32,33,22,31,42,25,22,21,23,20,33,30,31,29,33,29,28,21,16,24,31,31,31,29,29,24,28,28,23,28,28,26,15,27,24,31,32,34,24,26,26,32,32,31,31,35,34,20,14,36,35,25,35,14,44,23,18,22,20,39,24,26,30,12,28,34,23,27,9,33,33,12,33,32,32,31,39,20,22,30,15,32,34,21,22,28,29,24,24,19,38,11,28,29,21,25,29,31,28,28,30,29,30,30,25,25,43,36,35,27,28,21,24,16,22,21,36,32,19,28,29,27,28,30,21,23,23,29,31,32,21,29,28,31,21,22,29,27,29,28,29,29,37,23,25,36,36,38,21,32,32,32,23,25,34,36,20,21,22,32,8,30,26,31,34,30,29,26,29,29,37,37,32,32,35,35,35,32,30,33,30,30,22,21,28,30,29,9,15,27,32,32,31,33,35,34,25,23,40,40,39,39,37,22,21,31,35,23,24,34,34,31,28,20,33,30,29,28,33,35,34,22,35,28,28,26,28,39,40,34,31,24,25,23,23,36,35,34,33,33,27,28,37,16,26,28,31,30,24,27,32,34,35,30,39,39,34,27,15,39,38,16,31,34,28,19,27,38,32,20],"y":[24,24,38,38,6,3,3,8,8,8,6,6,6,9,9,5,5,5,11,11,5,5,5,11,11,12,12,13,13,13,11,11,50,9,9,11,11,17,13,13,13,15,15,25,42,31,31,25,25,7,7,3,3,54,5,5,5,5,5,5,5,11,11,31,31,21,21,21,2,2,2,16,16,6,12,12,12,9,9,9,7,2,2,4,4,10,10,10,2,2,2,42,42,5,5,17,17,19,19,1,5,2,2,8,8,7,8,8,9,9,1,1,5,5,4,4,4,12,12,11,5,5,5,0,3,3,3,0,3,10,10,10,0,2,7,7,7,5,5,5,6,6,6,3,4,4,4,4,11,3,3,3,2,0,0,0,19,11,11,10,10,10,10,8,1,3,3,5,5,5,5,5,5,15,8,8,8,8,3,5,3,3,0,0,3,5,5,0,0,2,2,43,1,0,0,27,27,27,27,40,44,44,65,36,36,13,13,3,0,1,1,13,13,13,3,3,0,2,2,1,1,0,38,38,33,33,16,16,16,14,14,14,16,16,16,52,52,48,36,36,36,60,14,49,7,14,14,42,0,0,0,3,3,2,3,3,0,0,8,8,0,49,7,7,7,1,25,21,21,24,24,63,0,3,43,43,22,0,0,10,18,18,18,48,0,1,1,3,3,3,17,17,3,3,3,38,38,20,20,20,4,4,4,30,24,24,22,20,2,0,0,0,4,0,0,3,21,21,23,23,23,15,15,5,5,33,33,35,35,30,30,11,11,11,20,20,28,28,28,71,71,76,76,5,49,49,8,8,41,21,21,32,40,40,41,41,46,46,56,44,44,0,4,3,3,6,6,4,4,4,5,35,35,35,2,2,63,34,34,38,38,33,11,59,11,11,34,34,7,43,43,31,31,45,27,27,10,10,44,44,10,10,3,3,8,24,24,24,24,24,4,4,1,1,12,12,6,6,19,19,5,5,30,30,0,0,0,32,32,0,0,28,28,0,10,10,2,2,2,13,13,9,9,9,9,1,1,1,1,3,3,3,3,9,9,3,3,1,1,0,0,11,11,12,12,31,31,41,41,7,7,42,4,12,12,12,32,32,32,7,7,7,5,5,5,5,17,17,30,10,10,13,13,58,31,31,41,41,41,36,26,26,1,1,1,1,1,1,1,3,3,3,3,4,8,8,10,3,3,3,3,11,11,2,2,2,2,2,3,3,4,4,13,13,5,5,5,5,6,6,17,17,15,15,6,6,11,16,6,6,13,13,4,4,12,12,12,38,8,8,28,51,5,5,15,15,17,17,17,17,23,0,10,10,10,16,16,20,10,10,10,10,50,14,14,14,14,2,7,7,28,2,2,1,0,0,11,13,13,9,9,17,2,2,2,18,19,46,6,6,6,21,21,22,22,60,4,4,4,2,2,2,14,14,14,9,5,5,2,2,2,40,6,6,6,17,17,57,5,5,5,5,5,5,13,13,15,43,4,0,15,15,15,7,7,7,9,2,2,2,8,8,3,3,44,20,20,4,6,6,6,6,26,26,26,40,40,11,11,11,44,44,50,6,3,3,3,2,2,6,6,6,6,10,10,0,0,0,7,7,7,4,4,4,23,23,46,5,5,7,7,7,31,30,6,6,6,10,10,5,8,3,3,3,31,31,19,10,10,10,10,33,28,30,1,5,2,2,2,2,11,11,11,11,11,11,0,0,0,0,12,12,12,12,12,21,21,18,18,24,6,6,9,10,10,10,6,16,16,40,10,10,10,12,12,13,13,13,20,20,22,0,3,3,9,9,4,4,4,11,25,25,9,9,9,9,9,34,35,16,16,16,41,54,6,6,4,4,4,4,5,5,5,5,5,9,9,9,18,18,11,11,16,51,3,26,26,36,36,8,8,36,27,27,23,23,43,43,40,4,4,4,20,20,11,11,2,9,9,9,9,4,4,11,10,10,5,5,17,17,33,0,0,0,0,31,31,8,8,8,8,24,24,15,15,15,13,13,13,13,12,12,12,12,40,40,11,11,11,11,40,40,35,35,35,19,41,41,46,46,19,19,19,19,1,0,0,1,1,11,11,11,3,3,3,3,3,9,9,9,8,8,16,10,10,10,12,12,12,8,4,4,4,10,10,11,11,11,10,10,10,10,10,10,12,12,12,12,43,5,5,5,5,20,20,45,2,2,26,26,20,20,24,24,3,3,3,14,14,7,7,7,7,7,16,16,16,21,21,17,17,17,7,7,7,12,12,12,6,6,5,5,5,4,4,4,7,7,7,10,10,7,7,7,19,19,12,12,12,41,46,46,2,3,3,3,5,5,5,5,8,8,8,8,2,0,5,5,5,33,34,28,4,2,2,2,11,11,14,14,8,8,8,21,21,18,18,18,7,44,44,3,3,3,2,2,2,7,7,33,33,8,8,8,8,14,14,14,15,44,40,9,9,38,20,20,3,3,3,3,31,31,17,17,17,47,47,16,16,4,4,4,4,8,8,8,3,3,3,46,16,16,4,4,4,9,9,6,6,6,0,0,4,4,4,5,5,5,5,5,8,8,8,8,8,4,4,8,8,8,4,4,4,43,8,8,3,3,3,3,16,16,4,4,4,4,5,30,30,9,9,9,2,2,2,2,2,5,6,6,6,6,3,3,3,5,5,5,25,25,3,3,0,8,32,32,3,3,3,4,4,4,14,26,26,1,1,1,1,15,15,2,2,2,32,12,12,12,4,4,10,10,5,6,15,15,15,66,7,7,7,7,7,6,6,6,5,5,5,25,25,1,1,1,16,16,3,3,4,4,4,1,1,1,3,3,3,3,3,2,2,1,1,13,13,5,5,5,1,1,13,13,13,13,13,13,0,8,8,13,1,1,1,30,30,30,6,7,7,6,6,4,4,4,17,17,17,17,17,33,33,31,31,31,25,25,7,7,31,11,27,27,18,18,7,19,34,34,10,10,52,2,2,34,34,9,9,9,9,34,34,10,10,10,17,17,17,18,18,42,42,18,18,2,2,2,2,3,3,3,3,2,2,4,4,3,3,6,6,6,11,11,7,7,7,7,15,15,15,15,6,6,6,18,18,10,10,30,30,6,6,6,6,20,12,12,12,8,8,7,7,28,28,6,6,8,8,7,7,7,20,20,12,12,7,19,19,4,4,4,8,8,4,4,3,3,7,10,13,3,3,3,4,4,6,6,6,7,7,21,21,1,1,6,6,4,3,3,8,8,3,2,2,14,10,10,11,19,1,1,1,1,2,2,2,2,4,3,3,3,4,4,3,3,18,18,9,9,19,19,19,12,12,23,23,19,19,20,20,3,3,4,4,6,6,6,8,15,15,14,14,9,15,15,4,4,5,7,7,40,40,40,0,0,4,4,13,13,13,20,20,3,3,10,10,6,6,20,5,5,31,12,12,6,6,17,2,27,11,11,11,2,35,35,28,1,1,4,4,5,5,9,9,10,10,20,20,20,4,4,4,2,1,1,1,17,17,14,14,8,8,8,8,14,14,14,13,13,13,0,0,0,7,7,7,13,13,12,12,15,15,15,7,7,7,21,21,14,1,1,1,31,31,3,3,18,18,18,9,9,8,8,8,3,3,5,5,10,10,13,13,13,15,15,15,1,1,0,0,0,14,5,5,5,14,14,14,5,5,5,23,23,9,9,9,11,11,11,9,9,2,2,8,8,4,4,4,8,8,5,5,5,1,1,2,10,10,10,8,8,8,6,6,4,4,4,6,6,6,0,0,0,3,3,12,12,18,18,8,8,8,10,10,10,13,13,13,4,4,4,4,3,3,3,9,9,6,6,6,23,3,3,3,5,5,9,9,0,0,24,1,1,10,10,10,4,4,4,4,9,9,9,16,3,45,45,10,10,14,26,33,32,50,15,15,7,7,32,32,28,28,14,14,44,48,48,4,4,4,1,23,23,23,20,20,20,23,23,31,31,31,31,15,15,28,17,17,17,10,10,10,38,20,20,40,40,53,45,36,2,2,2,9,9,32,32,0,1,1,35,35,35,30,30,5,5,5,34,12,12,2,2,3,25,25,4,4,50,50,5,24,24,17,17,54,54,35,35,24,24,46,46,0,38,52,52,32,58,27,35,35,31,31,49,25,52,52,51,21,21,21,6,6,37,37,46,42,42,32,67,10,10,51,51,60,60,25,25,2,43,36,13,13,52,52,21,21,21,11,11,11,4,4,20,20,44,17,17,23,23,42,42,0,46,46,19,19,52,17,17,35,35,3,53,0,0,43,5,5,48,26,26,26,24,24,1,1,33,33,17,17,23,12,12,8,8,8,19,8,8,8,26,26,26,26,44,44,66,22,22,22,59,19,19,27,27,42,42,42,16,16,10,10,23,23,5,5,23,23,13,13,36,36,10,10,10,0,34,16,24,24,15,15,25,25,29,29,8,8,8,8,8,49,49,7,7,36,36,23,23,7,7,19,12,12,44,40,25,25,41,36,9,9,5,5,54,11,11,35,35,17,17,18,18,5,5,16,16,16,8,8,7,3,30,30,12,12,36,36,15,15,15,15,2,2,21,4,16,5,5,65,3,34,20,26,22,0,0,60],"z":[74.1100006103516,71.1100006103516,64,50,68.379997253418,59.8600006103516,61.9000015258789,61.0999984741211,59.4199981689453,66.370002746582,76.5699996948242,71.2900009155273,81.0400009155273,65.7799987792969,68.5046691894531,71.3000030517578,77.2600021362305,73.3000030517578,57.8600006103516,56.5400009155273,71.0899963378906,65.2099990844727,64.4830322265625,56.2299995422363,57.7020606994629,67.0899963378906,64.4100036621094,49.5200004577637,59.5,60.2900009155273,64.75,67.2962951660156,55.5800018310547,68.5800018310547,67.4683303833008,67.7600021362305,75.3499984741211,63,62.5699996948242,62.5800018310547,53.7799987792969,44.2700004577637,57.0800018310547,48.7299995422363,67.5,47.7099990844727,51.5400009155273,68.8899993896484,61.304500579834,72.3600006103516,65.4800033569336,77.5999984741211,78.470588684082,48.75,64.129997253418,70.5199966430664,67.0899963378906,63.310001373291,63.8600006103516,56.2200012207031,62.3300018310547,58.439998626709,61.9967994689941,62.5900001525879,57.6300010681152,48.0699996948242,55.810001373291,62.7900009155273,54.3899993896484,55.4500007629395,58.2993545532227,53.9099998474121,54.9599990844727,62.7799987792969,81.4300003051758,74.3899993896484,75.4599990844727,61.1500015258789,66.5500030517578,56.2099990844727,72,69.7399978637695,69.8000030517578,79.8600006103516,72.1399993896484,56.1199989318848,60.3600006103516,61.9599990844727,72.6100006103516,66.0999984741211,68.5299987792969,48.9500007629395,49.0527763366699,73.0599975585938,75.5,74.5599975585938,58.8300018310547,61.8300018310547,62.1457138061523,78.1699981689453,58.4799995422363,74.25,77.5199966430664,70.6600036621094,63.4900016784668,81.2699966430664,68.3199996948242,59.6399993896484,69.3499984741211,74.9023818969727,64.0599975585938,68.2099990844727,74.7099990844727,77.8899993896484,67.8300018310547,73.6900024414062,84.0800018310547,47.3300018310547,39.4165534973145,81.2200012207031,63.1500015258789,67.879997253418,72.7859268188477,73.2200012207031,67.379997253418,72.2900009155273,70.9700012207031,66.7600021362305,70,61.6100006103516,66.9599990844727,70.0800018310547,59.4099998474121,78.0299987792969,75.1399993896484,69.4199981689453,70.9981842041016,55.0900001525879,65.370002746582,64.3099975585938,73.9300003051758,67.7699966430664,65.53125,80.25,65.25,75.870002746582,72.6800003051758,79.9100036621094,71.370002746582,67.7099990844727,67.4800033569336,65.5699996948242,74.2699966430664,66.7099990844727,68.8199996948242,65.5408554077148,68.0299987792969,64.0800018310547,60.9000015258789,65.5400009155273,72.7900009155273,65.4700012207031,73.0299987792969,63.1300010681152,68.0899963378906,61.6300010681152,68.7099990844727,55.9599990844727,57,72.4599990844727,68.6699981689453,60.2299995422363,66.2591171264648,67.3300018310547,68,67.879997253418,65.7600021362305,75.2200012207031,71.8899993896484,62.2400016784668,71.8499984741211,72.1500015258789,67.3300018310547,61.7900009155273,76.4599990844727,56.4099998474121,75.2699966430664,66.3099975585938,69.3899993896484,65,62.1699981689453,62.5,67,87.1999969482422,66.379997253418,74.4199981689453,69.5400009155273,57.2599983215332,66.5,62.4099998474121,60.8699989318848,59.310001373291,57.689998626709,63.8499984741211,72.1500015258789,55.7299995422363,59.3499984741211,69,51.0999984741211,61.9099998474121,58.5699996948242,72,62.2000007629395,69.6399993896484,67.5599975585938,72.0400009155273,65.7699966430664,71.9499969482422,66.4039993286133,56.5,54.7400016784668,71,64.25,58.7299995422363,56.7000007629395,64.4103469848633,58.689998626709,69.129997253418,68.1977386474609,55,69.5999984741211,57.96875,76.4199981689453,63.5499992370605,67.1493911743164,37.7999992370605,41.7200012207031,46.2700004577637,57.5800018310547,55.3600006103516,59.8300018310547,48,59.4599990844727,51.3300018310547,70.9700012207031,68.9400024414062,80.4700012207031,69,60.6100006103516,71.5599975585938,67.5999984741211,58.3800010681152,61.6599998474121,69.1800003051758,68.9100036621094,76.129997253418,62.2900009155273,59.4519996643066,72.9700012207031,77.1999969482422,69.2900009155273,65.9000015258789,54.4599990844727,55.6100006103516,48.8874092102051,70.120002746582,69.870002746582,74.8300018310547,81.1699981689453,70.7900009155273,65.8600006103516,47,77.7099990844727,76.2099990844727,52.2900009155273,68.4035034179688,53.9199981689453,75.0400009155273,72.8695678710938,73.370002746582,60.6199989318848,60.7099990844727,61.8300018310547,58.1800003051758,73.8099975585938,82.3000030517578,77.629997253418,85.2699966430664,78.8600006103516,76.0015411376953,68,63.7000007629395,71.4100036621094,68.5599975585938,73.4199981689453,48.7000007629395,47.2631568908691,54.3499984741211,55.6599998474121,62.6399993896484,70.4400024414062,68.9199981689453,72.75,62,62.1699981689453,43.2599983215332,60.1300010681152,53.7900009155273,67.6100006103516,73.3300018310547,72.7171401977539,78.5299987792969,63.9700012207031,64.5,59.8571434020996,63.9099998474121,57.7099990844727,64.1584014892578,65.870002746582,58.75,64.4300003051758,69.3099975585938,64.6100006103516,55.6199989318848,52.7611770629883,71.6800003051758,77.2099990844727,57.1100006103516,60.8899993896484,56.5800018310547,65.6600036621094,71.8600006103516,72.0299987792969,69.4499969482422,58,51.1100006103516,57.3300018310547,61.060001373291,81.2460021972656,55.0400009155273,41.9599990844727,44.7999992370605,39.0526313781738,81.5,60.0800018310547,47.5,69.4700012207031,65.8499984741211,65.1100006103516,68.6699981689453,63.9199981689453,62.7900009155273,60.2999992370605,64.5899963378906,62.3800010681152,74.8300018310547,69.379997253418,67.870002746582,66.4300003051758,66.4000015258789,67.9300003051758,72.3300018310547,71,70.6699981689453,72,67.3300018310547,75.1699981689453,60.5200004577637,49.5800018310547,59.9928588867188,50.1699981689453,44,63,62.939998626709,65.9000015258789,72.0999984741211,51.0999984741211,64.3499984741211,68.9199981689453,63.8199996948242,50.5900001525879,56,59.9000015258789,58.4300003051758,53.5699996948242,46.1749992370605,69.4199981689453,64.1600036621094,69,71.0599975585938,74.9199981689453,67.629997253418,58.5353584289551,56.189998626709,68.5999984741211,67.4444427490234,67.629997253418,68.4700012207031,72.7300033569336,53.8915786743164,64,53.7700004577637,61.7200012207031,67.1699981689453,65.4954147338867,58,49.9000015258789,67.2399978637695,71.3099975585938,73.7900009155273,74.0800018310547,65.1039962768555,75.9000015258789,70.4499969482422,59.9199981689453,64.1347351074219,49.3699989318848,50.9199981689453,65.0299987792969,58.3800010681152,64.8499984741211,57.0486831665039,64.8600006103516,50.5400009155273,67.370002746582,64.0899963378906,67.4100036621094,68.8300018310547,60.7280769348145,71.6699981689453,77.2200012207031,69.25,60.4599990844727,69.0500030517578,54.0499992370605,59.5477256774902,53.6199989318848,58.0999984741211,53.5179328918457,61.8899993896484,72.4199981689453,71.4499969482422,55.7900009155273,59.5999984741211,68.2399978637695,74.4499969482422,74.7099990844727,72.0899963378906,74.120002746582,62.189998626709,67.8199996948242,59.5900001525879,67.0943450927734,66.9400024414062,74.5599975585938,66.0599975585938,79.3000030517578,81.5999984741211,78.8199996948242,83.2900009155273,84.6800003051758,54.5099983215332,54.0800018310547,68.8300018310547,69.7600021362305,63.0400009155273,63.0699996948242,73.2600021362305,67.3300018310547,79.9499969482422,69.9100036621094,61.8499984741211,55,65.9700012207031,65.6900024414062,65.629997253418,69.1999969482422,45.2000007629395,54.1300010681152,60.9700012207031,65.0500030517578,60.5299987792969,69.5999984741211,83.3000030517578,53.6699981689453,66.2900009155273,68.2099990844727,63.439998626709,57.4700012207031,77.7099990844727,72.9300003051758,55.4300003051758,52.3800010681152,63.7599983215332,63.2900009155273,58.1500015258789,68.120002746582,74.2639999389648,46.7000007629395,78.379997253418,68.4300003051758,63.2999992370605,79.6900024414062,79.5500030517578,79.5100021362305,85.5638885498047,69.9100036621094,81.0599975585938,70.4793090820312,77.0500030517578,80.7399978637695,74.1800003051758,80.3052749633789,80.3600006103516,72.4899978637695,67.1800003051758,61.0999984741211,74.9400024414062,65.7300033569336,67.6100006103516,93.9300003051758,58.0400009155273,71.9499969482422,86.7699966430664,81.8175735473633,85.0699996948242,85.5199966430664,85.2399978637695,78.0299987792969,75.7900009155273,67.5999984741211,70.4000015258789,62.25,57.4099998474121,67.9400024414062,74.5400009155273,69.2099990844727,77.3499984741211,68.6100006103516,69.2699966430664,60.8400001525879,63.3699989318848,64.0500030517578,66.4300003051758,62.5,60.2799987792969,70.2399978637695,62.2000007629395,85.1900024414062,86.8099975585938,74.3600006103516,71.2200012207031,82.120002746582,75.6787109375,70.1399993896484,73.0500030517578,59.0971870422363,69.9400024414062,80.0999984741211,72.6900024414062,52.1100006103516,47.1800003051758,76.4599990844727,75.4968719482422,63.9099998474121,70.0400009155273,67.0400009155273,66.120002746582,68.1100006103516,70.8600006103516,58.5800018310547,83.4700012207031,58.1800003051758,59.0499992370605,59.8102684020996,63.4799995422363,67.4199981689453,64.879997253418,69.1500015258789,64.2200012207031,54.5200004577637,60.0963325500488,57.1800003051758,72,73.9000015258789,77.3000030517578,74.4800033569336,72.3199996948242,64.4300003051758,61.4099998474121,77,64.9700012207031,69.2699966430664,67,62.5299987792969,74.6699981689453,66,76.3600006103516,60.1300010681152,80.6699981689453,73.0363998413086,75.9700012207031,61,59.939998626709,68,67,65.7900009155273,44.689998626709,66.9599990844727,58.1599998474121,57.1199989318848,67.3300018310547,68.6117401123047,64.8099975585938,60.6599998474121,44,73.879997253418,69.4800033569336,73.4400024414062,71.1399993896484,78.129997253418,74.2600021362305,60.3199996948242,55.5900001525879,60.1599998474121,70,74.2799987792969,74.1999969482422,77.6800003051758,81.2600021362305,76.2200012207031,72.1999969482422,68.8600006103516,63.8400001525879,62.2599983215332,70.5899963378906,67.4100036621094,73,54.1800003051758,71.0299987792969,62.1313323974609,75.9199981689453,77.8600006103516,74.4349975585938,72.3199996948242,78.9400024414062,76,69.6999969482422,81.2699966430664,81.8199996948242,61.060001373291,53.810001373291,62.2900009155273,64.3600006103516,63.3199996948242,57.1199989318848,52,80,68.1999969482422,75,77.4800033569336,73.0305862426758,80.5299987792969,81.3300018310547,56,79.5800018310547,79.1399993896484,81,77.9199981689453,70.4300003051758,74,76.8118515014648,47.7299995422363,55.5299987792969,59.2778587341309,31.4500007629395,47,67.7099990844727,75.9700012207031,59.647777557373,63.8800010681152,62.2200012207031,48.5699996948242,60.8199996948242,75.0400009155273,80.0699996948242,75.966552734375,87.0500030517578,77.25,71.9000015258789,68.5500030517578,67.4800033569336,70.4725036621094,69.2099990844727,62.7400016784668,81,77.2600021362305,70.0899963378906,63.2599983215332,67.1500015258789,55.8400001525879,74.8899993896484,72.8600006103516,79.9971084594727,61.4500007629395,53.1304359436035,58.7900009155273,78,79.4000015258789,66.7699966430664,86.4100036621094,84.4199981689453,56,44,64.629997253418,70.5,69.5858306884766,77.5899963378906,65.3199996948242,75.5400009155273,80.8899993896484,67.1500015258789,74.2099990844727,65.7360000610352,68.879997253418,64.7699966430664,68.129997253418,81.0699996948242,74.8600006103516,71.620002746582,54.7718200683594,58.1599998474121,52.8199996948242,65.0699996948242,72.8099975585938,78.6999969482422,71.2300033569336,79.6800003051758,62.8600006103516,69.5366668701172,78.6500015258789,71.7711563110352,62.689998626709,60.5200004577637,70.1699981689453,69.8488922119141,62.8300018310547,66.1500015258789,67.9599990844727,57.939998626709,80.0400009155273,73.7900009155273,79.5685729980469,71.5899963378906,75.9599990844727,62.7000007629395,61.3636360168457,66.3199996948242,60.4300003051758,58,67.9400024414062,76.9700012207031,73.1399993896484,61.6100006103516,74.879997253418,68.2699966430664,78,69.5699996948242,65.1100006103516,72.0599975585938,69.5599975585938,66.9100036621094,61.6813774108887,63.0900001525879,54.4900016784668,74.9000015258789,74.3199996948242,64.0299987792969,73.5599975585938,62.2099990844727,59,73.2699966430664,63.0900001525879,69.4100036621094,72.5899963378906,69.3899993896484,73.129997253418,77.8199996948242,73.7055282592773,71.5500030517578,68.2200012207031,67.7572402954102,77.5100021362305,72.2200012207031,75.7900009155273,69.4700012207031,74.3199996948242,57,50.2099990844727,64.6600036621094,66.2699966430664,70.0951690673828,72.629997253418,79.8000030517578,73.5999984741211,73.9599990844727,81.2099990844727,81.4899978637695,76.6900024414062,76.3767623901367,73.870002746582,77.3899993896484,79.4599990844727,74.8499984741211,79.9700012207031,66,67.3199996948242,74.0299987792969,65.5,57.8199996948242,71.4599990844727,82.3942108154297,57.0699996948242,61.3300018310547,83.7699966430664,84.1999969482422,83.2900009155273,57.6500015258789,67.1800003051758,72.8099975585938,60.8499984741211,38.1800003051758,56.1100006103516,48.6500015258789,63.060001373291,71.129997253418,51.3800010681152,45.8199996948242,62.2200012207031,73.8899993896484,85.4800033569336,66.4800033569336,77.5199966430664,71.4300003051758,68.379997253418,78.8252182006836,72.5,58.8300018310547,76.3099975585938,68.9400024414062,73.974853515625,66.0899963378906,81.1399993896484,63,78.25,80.7900009155273,74.9700012207031,85.0500030517578,74.5400009155273,78.4599990844727,48.6699981689453,78.6100006103516,77.6399993896484,75,84.1800003051758,47.7000007629395,72.2600021362305,80.2399978637695,74.4400024414062,82.4000015258789,74.6933364868164,74.0599975585938,80.9300003051758,83.620002746582,83.9199981689453,79.9199981689453,72.2699966430664,80.2900009155273,74.8099975585938,72.7900009155273,73.5899963378906,71.6399993896484,72,68.8499984741211,59.1100006103516,54.0999984741211,75.7699966430664,85.0299987792969,83.0899963378906,75.2616119384766,70.3300018310547,54.7112503051758,78.0800018310547,68.6699981689453,73.120002746582,66.0899963378906,69.4499969482422,63.6699981689453,54.2099990844727,53.3699989318848,71.6900024414062,62.1599998474121,66,72.1548385620117,76.4700012207031,62,77.2399978637695,72.0899963378906,78.2399978637695,69.1500015258789,78.3199996948242,72.8399963378906,72.5199966430664,62.4700012207031,80.6500015258789,73.3099975585938,74.8978118896484,50.1699981689453,66.1100006103516,92.6900024414062,57.2799987792969,57.4300003051758,55.3600006103516,65.1699981689453,68.8300018310547,68.1999969482422,79.3399963378906,74.879997253418,83.6500015258789,62.3199996948242,62.5400009155273,75.4300003051758,66.0299987792969,73.6399993896484,66.379997253418,56.1199989318848,72.5899963378906,75.2600021362305,74.8899993896484,80.2900009155273,75.6783981323242,85.4400024414062,79.9599990844727,82.8099975585938,61.0299987792969,60.3800010681152,54.5800018310547,69.7303848266602,67.5199966430664,87.8300018310547,81.4300003051758,71.9300003051758,74,61.7799987792969,72.2200012207031,61.5400009155273,76.7900009155273,79.6100006103516,74.6500015258789,78,57.7099990844727,64.7399978637695,65.2399978637695,66.2099990844727,80.879997253418,74.9400024414062,78.0870971679688,79.1900024414062,69.2600021362305,76.3499984741211,79.9300003051758,72.3499984741211,85.5,70.9743270874023,70.1999969482422,64.2900009155273,66.4325714111328,55.6800003051758,59,82.1699981689453,73.1399993896484,73.6900024414062,78.5999984741211,69.0400009155273,76.9599990844727,76.3499984741211,86.2099990844727,85.955924987793,80.6699981689453,80.75,84.7600021362305,76.8899993896484,86.2900009155273,81.4800033569336,84.6900024414062,76.2099990844727,68.6999969482422,79.1999969482422,76.6916656494141,75.3499984741211,71.4199981689453,64.4800033569336,78.2699966430664,74.8899993896484,64.2099990844727,74.0561141967773,69.5899963378906,64.879997253418,66.5148391723633,53.7099990844727,60.4099998474121,63.5208702087402,69.8499984741211,66.370002746582,64.8899993896484,66.907096862793,78.1500015258789,76.8300018310547,77.8199996948242,75.2891464233398,80,81.0299987792969,75.2099990844727,75.5199966430664,64.0899963378906,64.3199996948242,61.4500007629395,62.2799987792969,71.0899963378906,50,70.870002746582,43.560001373291,65.620002746582,83.620002746582,89.3300018310547,75.7399978637695,59.8400001525879,59.9700012207031,59.3199996948242,55.3600006103516,69.370002746582,67.0800018310547,62.939998626709,61.060001373291,59.6399993896484,73.1699981689453,74.3300018310547,59.731819152832,60.4700012207031,37.5800018310547,45.9563636779785,69.9599990844727,77.4599990844727,74.1791305541992,79.0999984741211,82.9300003051758,81.8099975585938,69.0800018310547,65.6699981689453,71.379997253418,72.7831573486328,81.7099990844727,76.7699966430664,73.8899993896484,79.4089736938477,65.7399978637695,68.8300018310547,69.5807723999023,69.2799987792969,85.9400024414062,50,69.3000030517578,66.4300003051758,57.810001373291,67.9599990844727,56,77.25,71.2600021362305,80.5999984741211,67.0599975585938,47.4500007629395,46.5499992370605,62.189998626709,68.0400009155273,64.629997253418,64.6900024414062,58.6800003051758,72.4400024414062,54.2592582702637,80.2099990844727,74.7099990844727,81.4100036621094,79.5407180786133,77.620002746582,66.0800018310547,63.4812126159668,77.3399963378906,67.5400009155273,75,27.6900005340576,70.9199981689453,73.3000030517578,74.8300018310547,70.9000015258789,69.6999969482422,84.379997253418,77.7600021362305,70.0999984741211,65.6699981689453,65.0673370361328,87.7900009155273,86.5500030517578,71.4599990844727,68.3099975585938,77.5400009155273,87.0199966430664,84.9276885986328,75.4700012207031,73.379997253418,82.0543746948242,74.3199996948242,66.1900024414062,66.1699981689453,72.25,65.8970336914062,76.8499984741211,62.7900009155273,74.3099975585938,71.3099975585938,76.7399978637695,80.2300033569336,78.1800003051758,67.7200012207031,64,76.4100036621094,71.379997253418,71.1800003051758,78.7699966430664,80.120002746582,81.6600036621094,71.2399978637695,73.2399978637695,73.75,79.4100036621094,78.0017166137695,75.5699996948242,73.3099975585938,66.5599975585938,53.5,68.3199996948242,68.4400024414062,70.2588882446289,80.4499969482422,81.6900024414062,77.6900024414062,77.5500030517578,72.5800018310547,76,75.8499984741211,71.0299987792969,72.1500015258789,72.4100036621094,83.0299987792969,83.25,86.1169204711914,68.8300018310547,70.0800018310547,74.0899963378906,54.9099998474121,64.3899993896484,76.8499984741211,79.1089248657227,82.2799987792969,59.689998626709,74.8600006103516,74.4599990844727,64.370002746582,68.7900009155273,73.6770935058594,65.6800003051758,71.5800018310547,78.0968399047852,76.0999984741211,55.9700012207031,53.0800018310547,74.6100006103516,68.4700012207031,66.5699996948242,70.1945724487305,67.2099990844727,64.903434753418,82.4499969482422,86.0699996948242,79.879997253418,62.7000007629395,67.879997253418,69.5299987792969,66.5445175170898,76.879997253418,71.370002746582,60.2200012207031,68.2525024414062,76.8199996948242,58.810001373291,68.3300018310547,71.2900009155273,70.6354522705078,42.3199996948242,76.5699996948242,79.8099975585938,74.6061325073242,73.1399993896484,62.7299995422363,65.6100006103516,70.3099975585938,65.5199966430664,77,77.870002746582,81.8300018310547,57.3300018310547,52.5686950683594,73.6500015258789,78.1900024414062,64.4700012207031,74.6699981689453,68.7900009155273,85.870002746582,79.3899993896484,59.7099990844727,56.3199996948242,62.2999992370605,79.5500030517578,72.1699981689453,76.129997253418,79.9700012207031,74.2399978637695,72.1600036621094,74.1399993896484,73.7200012207031,65.4199981689453,73.2200012207031,58.5699996948242,69.1800003051758,60.9199981689453,79.4431991577148,62.3800010681152,61.5200004577637,67.9633331298828,82.9400024414062,77.7900009155273,72.5999984741211,75.4199981689453,67.4100036621094,67.4199981689453,70.1900024414062,67.0022201538086,79,63.9599990844727,64.6100006103516,73.2699966430664,87.6399993896484,78.9400024414062,82.7399978637695,61.8400001525879,67.6100006103516,58.3499984741211,81.8099975585938,73.7300033569336,77.7099990844727,52.7099990844727,55.2900009155273,66.3300018310547,87.6699981689453,79.1399993896484,48.6500015258789,59.2700004577637,51.7299995422363,62.8300018310547,65.7300033569336,60.6399993896484,57.2099990844727,54.0999984741211,63.3400001525879,50.8046684265137,47.939998626709,38.75,67.5999984741211,67.4300003051758,74,63,54.7799987792969,54.5400009155273,62.4500007629395,65.954345703125,62.0699996948242,51.4799995422363,60.7599983215332,61.310001373291,60.7900009155273,66.6100006103516,32.0499992370605,85.1699981689453,73.9199981689453,39.8899993896484,55.9995002746582,70.3199996948242,74.3600006103516,66.7200012207031,72.9680023193359,55.1599998474121,48,54.0299987792969,56.4000015258789,56.9199981689453,61.6699981689453,64.620002746582,61.4500007629395,56.8199996948242,56.060001373291,47.0900001525879,49.3699989318848,54.4599990844727,62.6699981689453,75.4499969482422,76.5805587768555,66.5199966430664,62.4199981689453,71.629997253418,82.1551971435547,72.6500015258789,76.1699981689453,71.5,72.1500015258789,82.3199996948242,76.0800018310547,72.129997253418,70.7399978637695,81.0599975585938,74.0500030517578,76.237060546875,64.129997253418,60.4099998474121,77.2600021362305,69.9700012207031,62.3800010681152,63.4300003051758,82.2099990844727,74.129997253418,79.8000030517578,67.4400024414062,83.5400009155273,78.9599990844727,86.379997253418,62.8199996948242,67.2900009155273,59.2799987792969,67.7699966430664,58.1500015258789,60.7000007629395,66.8000030517578,69.7799987792969,55.310001373291,67.3140640258789,69.6500015258789,55.2599983215332,62.9000015258789,92.3399963378906,77.8000030517578,61.875,67.9100036621094,63.689998626709,45.3800010681152,48.2799987792969,67.1399993896484,62.6399993896484,69.4000015258789,71.4800033569336,71.2099990844727,68.370002746582,73.583610534668,48.0400009155273,62.689998626709,57.6199989318848,50.5299987792969,58.6300010681152,55.8699989318848,58.5,73.370002746582,70.9300003051758,74.6588897705078,62.2599983215332,70.5137939453125,68.0400009155273,68.8399963378906,67.6900024414062,73.5329437255859,62.9500007629395,66.370002746582,64.1600036621094,67.0299987792969,68.4599990844727,69.651725769043,67.7699966430664,67.9700012207031,61.8300018310547,72.879997253418,65.5,68.1500015258789,69.8499984741211,57.0699996948242,55.439998626709,66.5100021362305,67.4899978637695,68.3899993896484,75.3300018310547,72,72.5400009155273,80.7799987792969,64.5,67.0800018310547,60.1699981689453,75.6999969482422,67.9476165771484,64.8199996948242,70.6900024414062,71.268669128418,69.6699981689453,64.620002746582,78.3899993896484,79.9300003051758,78.2300033569336,78.370002746582,71.0800018310547,74.7760009765625,83.1600036621094,74.9400024414062,69,81.5899963378906,79.1999969482422,86.25,77.8199996948242,70.6999969482422,76.9199981689453,81.3177108764648,60.4099998474121,71.0899963378906,72.2399978637695,67.5242080688477,76.6399993896484,65.9199981689453,74.3600006103516,71.2600021362305,64.5699996948242,75.0299987792969,83.2437896728516,59,54.2299995422363,58.6399993896484,46.2857131958008,68.129997253418,78.3499984741211,76.5899963378906,93.7300033569336,82.2200012207031,78.6800003051758,76.2779312133789,85.3499984741211,62.1199989318848,70.9504776000977,66.2200012207031,68.2900009155273,77.2699966430664,77.9599990844727,80.0800018310547,69.1500015258789,80.8928604125977,89.1600036621094,63.2099990844727,69.75,67.8300018310547,60.7000007629395,66.4752349853516,85.7900009155273,84.7699966430664,79.0800018310547,83,67.4100036621094,71.9300003051758,71.0299987792969,83.4199981689453,65.3099975585938,77.879997253418,68.5,78.7699966430664,75.1399993896484,66.9199981689453,74.3199996948242,67.3600006103516,66.3300018310547,59.5,45.6699981689453,65.5699996948242,77.6104354858398,81.8399963378906,78.3000030517578,64,66.1500015258789,52.1699981689453,63,56,52.5185203552246,82.4800033569336,57.560001373291,54.8654556274414,46.5,70.5599975585938,69.3913040161133,78.8000030517578,83.5100021362305,75.3499984741211,85.4309539794922,62.1599998474121,61.3800010681152,61.0800018310547,60.4099998474121,63.8600006103516,62.5900001525879,54.0637130737305,69.3099975585938,79.9599990844727,69.8282775878906,71.8199996948242,78.9199981689453,80.3300018310547,72.3000030517578,48.7900009155273,47.0519981384277,63.939998626709,69.5,68.1100006103516,67.8899993896484,82.2699966430664,88.6900024414062,75.7399978637695,57.1399993896484,48.4300003051758,61.7799987792969,55.3199996948242,56.9700012207031,75.0699996948242,75.5400009155273,68.0449981689453,69.5,79.4400024414062,83.8748397827148,62.4599990844727,62.5,64.3600006103516,62.0999984741211,72.1100006103516,65.2099990844727,63.0412483215332,71.4100036621094,56.5900001525879,61,55.75,50.1500015258789,62.4700012207031,80.0299987792969,84.3600006103516,80.7399978637695,63.6599998474121,49.8466682434082,68.7099990844727,73.1900024414062,66.629997253418,73.0400009155273,78.4000015258789,59.8600006103516,66.663330078125,56.6300010681152,69.0899963378906,51.9300003051758,70.75,73.379997253418,57.6599998474121,61.7900009155273,63.1599998474121,73.4199981689453,67.0599975585938,56.7099990844727,78.6255569458008,64.5500030517578,68.9599990844727,69.885383605957,76.3199996948242,76.5199966430664,85.2900009155273,83,86.8399963378906,55.6800003051758,64.870002746582,79,71.9300003051758,52.5200004577637,38.8300018310547,45.2299995422363,59.75,56.3699989318848,78.3724975585938,64.5500030517578,65.129997253418,65.2699966430664,67.7699966430664,61.9199981689453,68.2399978637695,65.5500030517578,66.2200012207031,59.5400009155273,58.4734992980957,77.7300033569336,77,70.7300033569336,70.5400009155273,82.2900009155273,76.6600036621094,74.1304779052734,66.5199966430664,70.3099975585938,63.6699981689453,66.7799987792969,58.9500007629395,70.5699996948242,86.0400009155273,85.9100036621094,60.1399993896484,58.9700012207031,70.1800003051758,56.7799987792969,65.5500030517578,74.4100036621094,78.9400024414062,79.0538482666016,76.2099990844727,72.5299987792969,69,71.3199996948242,75.6100006103516,76,74.9100036621094,72.879997253418,78.2799987792969,71.5299987792969,70.4700012207031,52.3899993896484,53.7400016784668,82.0500030517578,69.3300018310547,76.8600006103516,66.2699966430664,69.3426666259766,69.1100006103516,63.5400009155273,79.3300018310547,56.6500015258789,55.6800003051758,57.4799995422363,65,60.3600006103516,64.3600006103516,68.7600021362305,82.0599975585938,82.629997253418,79.870002746582,56.7900009155273,65.3375778198242,65.5,60.3600006103516,57.2763633728027,56.1199989318848,71.9700012207031,72.7099990844727,74.4155197143555,65.7900009155273,72.4300003051758,73.3000030517578,73.6100006103516,70.2200012207031,75.7799987792969,64.4199981689453,77.8000030517578,79.629997253418,56.9599990844727,63.4199981689453,53.3300018310547,76.7600021362305,69.4599990844727,73.8300018310547,72.2799987792969,57.1599998474121,70.1399993896484,65.879997253418,54.5200004577637,66.9700012207031,62,56.9599990844727,66.3199996948242,62.4599990844727,70.7600021362305,66.5800018310547,67.4800033569336,68.8600006103516,58.0900001525879,63.1800003051758,60.8537139892578,71.7900009155273,69,64.2200012207031,67.7799987792969,69.9599990844727,66.879997253418,76.5500030517578,65.5,75.2900009155273,55.8400001525879,66.5158615112305,62.1699981689453,63,63.8586196899414,67.8499984741211,58.4799995422363,60.2700004577637,55.4848289489746,58.6300010681152,61.9599990844727,58.8899993896484,65.5500030517578,60.2741394042969,62.6399993896484,52.7900009155273,57.2900009155273,53.5400009155273,70.9599990844727,68.3199996948242,66.5899963378906,58.7999992370605,63.7900009155273,56.4300003051758,59.7099990844727,64.5500030517578,54.0800018310547,51.5900001525879,61.6399993896484,66.1800003051758,64.9499969482422,56.0499992370605,47.2200012207031,55.8800010681152,70.8300018310547,69.9700012207031,65.2200012207031,64.9300003051758,68.879997253418,71.0870590209961,79.3600006103516,61.75,59.8600006103516,77.1999969482422,74.879997253418,49,40.6599998474121,45.8407707214355,77.5699996948242,80.8099975585938,73.2300033569336,73.75,73.129997253418,67,75.5599975585938,67.1399993896484,64.1399993896484,75.8300018310547,63.8499984741211,72.6800003051758,73.2666702270508,65.9700012207031,65.2900009155273,41.2999992370605,50.3240013122559,65.8300018310547,71.2799987792969,64.195198059082,76.4100036621094,73.5,65.0599975585938,62.6599998474121,66.6399993896484,50.5800018310547,61.5499992370605,61.8499984741211,47,62.9677429199219,62.7900009155273,66.2300033569336,73.0500030517578,45.6300010681152,55,52.1599998474121,76.1999969482422,59.7400016784668,48.0499992370605,58.6699981689453,58.2400016784668,58.0400009155273,51.189998626709,50.5,63.2173919677734,49.6699981689453,72.5800018310547,64.5599975585938,72.3300018310547,64.3000030517578,59.5516128540039,65.8899993896484,52,50.3499984741211,63.3699989318848,73.1399993896484,70.129997253418,53,68.3099975585938,74.0699996948242,70.9400024414062,61.9581832885742,54.1100006103516,43.6800003051758,61,68.879997253418,76.629997253418,60,55.5999984741211,62.1599998474121,65.8300018310547,62.2799987792969,50.9199981689453,62.5400009155273,75.8199996948242,71.7661514282227,72.8899993896484,73.2300033569336,70.0379333496094,72.1699981689453,78.629997253418,55.2400016784668,63.8300018310547,60.1500015258789,54.6300010681152,59.6758079528809,62.3199996948242,51.7099990844727,48.9099998474121,45.0499992370605,73.5,70.0500030517578,52.529411315918,73.5400009155273,74.5,41,67,75.3600006103516,43.8800010681152,46,62.9199981689453,62.4199981689453,66.6100006103516,61.8600006103516,51.8600006103516,62.6699981689453,51.1399993896484,71.6500015258789,57.4599990844727,57.3300018310547,59.0400009155273,55.8899993896484,59.2099990844727,64.6100006103516,59.9599990844727,62.7099990844727,54.844799041748,70.1399993896484,67.3600006103516,65.5599975585938,70,67.379997253418,64.5699996948242,63.3899993896484,66.9213180541992,53.1300010681152,57,56.7799987792969,54.2200012207031,55.2900009155273,60.7099990844727,56,58.1399993896484,55.5800018310547,50.4199981689453,56.4300003051758,71.4800033569336,60.0999984741211,69.1864318847656,42.5900001525879,57.7400016784668,57.5800018310547,57.7700004577637,59.9300003051758,70.2399978637695,64.7699966430664,68.7315368652344,61.060001373291,70.6699981689453,57.6300010681152,60.2400016784668,68.0999984741211,65.96875,78.4700012207031,73.4700012207031,47.2400016784668,54.4075012207031,54.1599998474121,59.2599983215332,71.4700012207031,70.5038909912109,67.6800003051758,60.8300018310547,63.560001373291,70.5,57.9300003051758,59.9099998474121,54.9000015258789,55.7200012207031,66.6100006103516,67.3412475585938,69.5,75.0299987792969,60.1399993896484,52.0400009155273,58.0499992370605,60.5400009155273,64.8600006103516,61.8600006103516,64.6883773803711,48.2000007629395,63.8300018310547,62.1699981689453,57.4500007629395,64.8300018310547,59.9599990844727,65.9700012207031,58.9700012207031,77.0999984741211,68.6071395874023,37.75,53.9599990844727,46.0400009155273,58.0400009155273,58.3899993896484,57.5499992370605,68.2796859741211,52.1199989318848,63.2200012207031,73.9000015258789,81.370002746582,75.7799987792969,61.6776008605957,64,62.4700012207031,71.6524353027344,44.1699981689453,44.7099990844727,71.0400009155273,74.8783340454102,72.3499984741211,71.6999969482422,68.5999984741211,72.5299987792969,74.4700012207031,71.5800018310547,66.8448486328125,67.129997253418,67.7099990844727,45,69.5,63.7000007629395,53.810001373291,52.0999984741211,55.9000015258789,59.4300003051758,54.4485015869141,61.8899993896484,67.7399978637695,68.1699981689453,67.7699966430664,70.7600021362305,72.9700012207031,69.0599975585938,93.2699966430664,64.5999984741211,69.1800003051758,79.8199996948242,49,88.7200012207031,66.1500015258789,71,68.4700012207031,70.2699966430664,70.0999984741211,69.5299987792969,50.3699989318848],"type":"scatter3d","mode":"markers","name":"1 dot = 1 class","opacity":0.6,"marker":{"color":"blue","size":1.5,"hoverinfo":"skip","line":{"color":"rgba(31,119,180,1)"}},"error_y":{"color":"rgba(31,119,180,1)"},"error_x":{"color":"rgba(31,119,180,1)"},"line":{"color":"rgba(31,119,180,1)"},"frame":null},{"colorbar":{"title":"avgmath<br />surf","ticklen":2,"len":0.5,"lenmode":"fraction","y":1,"yanchor":"top"},"colorscale":[[0,"red"],[1,"yellow"]],"showscale":true,"z":[[70.3027742470965,73.0982235151174],[44.4947880876703,47.2902373556912]],"type":"surface","x":[5,44],"y":[0,76],"name":"3D regression plane","opacity":0.7,"cauto":false,"frame":null}],"highlight":{"on":"plotly_click","persistent":false,"dynamic":false,"selectize":false,"opacityDim":0.2,"selected":{"opacity":1},"debounce":0},"shinyEvents":["plotly_hover","plotly_click","plotly_selected","plotly_relayout","plotly_brushed","plotly_brushing","plotly_clickannotation","plotly_doubleclick","plotly_deselect","plotly_afterplot","plotly_sunburstclick"],"base_url":"https://plot.ly"},"evals":[],"jsHooks":[]}</script> --- class:inverse # Task 1 (10 minutes) Let's analyse the regression results using **reading** score as the dependent variable. 1. Load the data from [here](https://www.dropbox.com/s/wwp2cs9f0dubmhr/grade5.dta?dl=1) using the `read_dta()` function from the `haven` package. Assign it to an object `grades`. 1. Regress `avgverb` on `classize` and `disadvantaged`. Assign the output to a new object `reg`. 1. Look at the coefficients of this regression by running `reg$coefficients`. How do they compare with the math score regression coefficients? 1. What are the other available variables that we may add in the regression? * Run the regression with all these variables and assign it to `reg_full`. * Look at the coefficients. * Discuss all coefficients: sign and magnitude. --- # No Perfect Collinearity There is one condition to satisfy to add regressors to the model: > Any additional variable needs to add **at least *some* new information**. -- In other words, regressors **cannot be perfectly collinear**, i.e. not linear combinations of one another: $$ x_2 \neq ax_1 + b $$ -- Even if not perfectly correlated, the individual effects of highly correlated regressors are hard to disantangle. -- Note that this implies that the number of observations has to be greater than the number of independent variables. --- class: inverse # Task 2 (7 minutes) Still need to think about this perfect collinearity issue? Let's run a regression where there is perfect linear dependence between regressors. 1. Load the *STAR* data from [here]("https://www.dropbox.com/s/bf1fog8yasw3wjj/star_data.csv?dl=1") and assign it to an object called `star_df`. Keep only cases with no `NA`s with the following code: `star_df <- star_df[complete.cases(star_df),]` Keep only second graders and small and regular class groups. 1. In the same dataset, add a dummy variable `star_df$small` equal to `TRUE` if students are in a small class and `FALSE` if they are in a regular class. In the same way, add a dummy variable `star_df$regular` equal to `TRUE` if students are NOT in a small class and to `FALSE` if they are in a small class. Clearly these two variables are perfectly collinear. * Regress `small` on `math`. What is the average predicted `math` score of students in a small class? * Regress `regular` on `math`. What is the average predicted `math` score of students in a regular class? * Regress `small` and `regular` on `math`. What do you notice? Can you explain why? --- # Adjusted `\(R^2\)` * Not of great importance but because it is so widely reported you just need to know it. -- * By construction, `\(R^2\)` will always increase when a new regressor is added to the regression. -- * The *adjusted `\(R^2\)`* imposes a penalty for adding regressors to the model. The details are not crucial as in the vast majority of cases the `\(R^2\)` and the adjusted `\(R^2\)` are pretty similar. --- # Ommitted variable bias (OVB) * ***ommitted variable bias***: Omitting important control variables from the regression model * This renders the coefficient for your regressor of interest unreliable. -- * Recall our previous example of class size and disadvantaged students. Not taking into account the `disadvantaged` variable led to a OVB for our class size estimate. -- * The formula for OVB is: `$$\textrm{OVB} = \{ \textrm{Relationship between } disadvantaged_i \textrm{ and } classize_i \} \\ * \{ \textrm{Effect of } disadvantaged_i \textrm{ in multiple regression} \}$$` -- * From this formula you obtain both the *mangitude* of OVB and its *sign* (positive/negative). -- * The relationship between `disadvantaged` and `classize` is negative, and the effect of `disadvantaged` in the multiple regression is negative, therefore the OVB is positive. In other words, ommitting `disadvantaged` from the regression leads to an inflated coefficient for class size. --- # Variations from the baseline model Depending on the dataset and the relationships between the variables of interest, you may need to move away from the baseline model. * We will focus on 3 important variations: * ***Standardized regressions*** * ***Log models*** * ***Interactions between regressors*** -- * In each case, the way we estimate these coefficients does not change (i.e OLS). -- * However, the way we interpret our coefficients `\((b_1, b_2,..., b_k)\)` does change! --- # Standardized Regression Let's define what *standardizing* a variable means. > ***Standardizing*** a variable `\(z\)` means to *demean* the variable and to divide the demeaned value by its own standard deviation: $$ z_i^{stand} = \frac{z_i - \bar z}{\sigma(z)}$$ where `\(\bar z\)` is the mean of `\(z\)` and `\(\sigma(z)\)` is the standard deviation of `\(z\)`, i.e. `\(\sigma(z) = \sqrt{\textrm{VAR}(z)}\)`. * Why would we do that in the first place? -- * Intuitively, standardizing **puts variables on the same scale** so we can compare them. * In our class size and student performance example, it will help to interpret: * The **magnitude** of the effects, * The **relative importance of each variable**. --- # Standardizing Regression: Interpretation * If the dependent variable `\(y\)` is standardized: -- * By definition, `\(b_k\)` measures the predicted change in ** `\(y^{stand}\)` ** associated with a one unit increase in `\(x_k\)`. * If `\(y^{stand}\)` increases by one, it means that `\(y\)` increases by one standard deviation. So `\(b_k\)` measures the change in `\(y\)` **as a share of `\(y\)`'s standard deviation**. -- * If the regressor `\(x_k\)` is standardized: -- * By definition, `\(b_k\)` measures the predicted change in `\(y\)` associated with a one unit increase in ** `\(x_k^{stand}\)` **. * If `\(x_k^{stand}\)` increases by one unit, it means that `\(x_k\)` increases by one standard deviation. So `\(b_k\)` measures the predicted change in `\(y\)` **associated with an increase in `\(x_k\)` by one standard deviation**. --- class:inverse # Task 3: To do at home (7 minutes) Let's go back our `grades` dataset. These are the estimates we got from regressing the math test score on the full set of regressors. ``` ## (Intercept) classize disadvantaged school_enrollment ## 70.31886504 0.02524502 -0.34728316 0.01834627 ## female religious ## -1.20818830 1.15754786 ``` 1. Create a new variable `avgmath_stand` equal to the the standardized math score. You can use the `standardize()` function from the `jtools` package or do it by hand with base `R`. 1. Run the full regression using the standardized math test score as the dependent variable. Interpret the coefficients and their magnitude. 1. If you were to pick the most influential variable on the math score, what would it be? 1. Add the standardized variables for each *continuous* regressor as `<regressor>_stand`. * Would it make sense to standardize the `religious` variable? 1. Regress `avgmath_stand` on the full set of standardized regressors and `religious`. Discuss the relative influence of the regressors. --- # Log Models * The models we have seen so far can be called ***level-level*** specifications. Both the dependent and the independent variables have been measured in level. -- * This *level* can be: euros, years, number of students,... and even percentage. -- * Taking the log of the dependent and/or the independent variable(s) leads us to define 3 other types of regressions: * ***Log - level***: `\(\quad log(y_i) = b_0 + b_1 x_{1,i} + ... + e_i\)` * ***Level - log***: `\(\quad \textrm{y}_i = b_0 + b_1 log(x_{1,i}) + ... + e_i\)` * ***Log - log***: `\(\quad log(y_i) = b_0 + b_1 log(x_{1,i}) + ... + e_i\)` --- # Log Models: Interpretation Here is a table that summarise how to interpret regressor coefficients in each case: | Model | Equation | Interpretation of `\(b_1\)` | |--------------------|:---------:|:-----------------------------------:| | Level - Level | `\(y = b_0 + b_1 x_{1} + e\)` | .small[**One unit** increase in ] `\(x_1\)` .small[ is associated,</br> on average, with ] `\(b_1\)` .small[**unit change** in y] | | Log - Level | `\(log(y) = b_0 + b_1 x_{1} + e\)` | .small[**One unit** increase in ] `\(x_1\)` .small[ is associated,</br> on average, with] `\(b_1\)` .small[ **percent change** in y] | | Level - Log | `\(y = b_0 + b_1 log(x_{1}) + e\)` | .small[**One percent** increase in ] `\(x_1\)` .small[ is associated,</br> on average, with] `\(b_1/100\)` .small[**unit change** in y] | | Log - Log | `\(log(y) = b_0 + b_1 log(x_{1}) + e\)` | .small[**One percent** increase in ] `\(x_1\)` .small[ is associated,</br> on average, with] `\(b_1\)` .small[**percent change** in y] | * It looks like cooking recipes but of course it can be derived with some calculus. --- # When do we use log models? There are several reasons why we would want to use log models: -- * Account for ***non linearity*** in the relationship between `\(y\)` and `\(x\)`. -- * Interpret coefficients as <a href="https://en.wikipedia.org/wiki/Elasticity_(economics)">***elasticities***</a> which play a central role in economic theory. -- * Limit the influence of ***outliers*** (due to the concavity of the *log* function). --- # Interacting Regressors * We interact two regressors when we believe the effect of one depends on the value of the other. * *Example:* The returns to education on wage vary by gender. -- * In practice, if we interact `\(x_1\)` and `\(x_2\)`, we would write our model like this : `$$y_i = b_0 + b_1 x_{1,i} + b_2 x_{2,i} + b_3x_{1,i}*x_{2,i} + ... + e_i$$` -- * The interpretation of `\(b_1\)`, `\(b_2\)`, and `\(b_3\)` will depend on the type of `\(x_1\)` and `\(x_2\)`. -- * Let's focus on the cases where one regressor is a dummy/categorical variable and the other is continuous. * It will give you the intuition for the other cases: * Both regresors are dummies/categorical variables, * Both regresors are continuous variables. --- # Interacting Regressors * Let's go back to the *STAR* experiment data. -- * How does the effect of being in a small vs regular class vary with the experience of the teacher? -- * Our regression model becomes: $$ \textrm{score}_i = b_0 + b_1 \textrm{small}_i + b_2 \textrm{experience}_i + b_3 \textrm{small}_i * \textrm{experience}_i + e_i$$ -- Effect of small class with teacher with 10 years of experience? -- `\(\mathbb{E}[\textrm{score}_i | \textrm{small}_i = 1 \textrm{ & experience}_i = 10] = b_0 + b_1 + b_2*10 + b_3*10\)` -- `\(\mathbb{E}[\textrm{score}_i | \textrm{small}_i = 0 \textrm{ & experience}_i = 10] = b_0 + b_2*10\)` -- `\(\begin{split} \mathbb{E}[\textrm{score}_i &| \textrm{small}_i = 1 \textrm{ & experience}_i = 10] - \mathbb{E}[\textrm{score}_i | \textrm{small}_i = 0 \textrm{ & experience}_i = 10] \\ &= b_0 + b_1 + b_2*10 + b_3*10 - (b_0 + b_2*10) \\ &= b_1 + b_3*10 \end{split}\)` --- # Interacting Regressors Running the regression for the `math` score (for all grades), we obtain: ```r lm(math ~ small+ experience + small*experience, star_df) ``` ``` ## ## Call: ## lm(formula = math ~ small + experience + small * experience, ## data = star_df) ## ## Coefficients: ## (Intercept) smallTRUE experience ## 534.1919 15.8906 1.3305 ## smallTRUE:experience ## -0.3034 ``` ***Interpretation:*** -- * The interaction term allows the impact of being in a small class to vary with the experience of the teacher. -- * In particular, we still observe a positive impact of being in a small class on math score, * but this effect is decreasing in the experience of the teacher. --- # Interacting Regressors: Visually <img src="chapter4_files/figure-html/unnamed-chunk-18-1.svg" style="display: block; margin: auto;" /> --- # Interacting Regressors: Visually by grade <img src="chapter4_files/figure-html/unnamed-chunk-19-1.svg" style="display: block; margin: auto;" /> --- # Wages, Education and Gender * Let's use these new *tools* to investigate the relationship between wages, education and gender. -- * We will use data from the [**C**urrent **P**opulation **S**urvey (CPS)](https://www.census.gov/programs-surveys/cps.html), the U.S. government's monthly survey of unemployment and labor force participation. * We'll use a sample of 534 individuals from the May 1985 CPS available in the `AER` package. -- .pull-left[ <img src="chapter4_files/figure-html/unnamed-chunk-20-1.svg" style="display: block; margin: auto;" /> ] -- .pull-right[ <img src="chapter4_files/figure-html/unnamed-chunk-21-1.svg" style="display: block; margin: auto;" /> ] --- # Wages, Education and Gender * So, wages are higher for men and higher for educated people, well, nothing really new. -- * But, are the returns to education the same for women and men? * In other words, how does the gender gap evolve for different levels of education? -- .pull-left[ <img src="chapter4_files/figure-html/unnamed-chunk-22-1.svg" style="display: block; margin: auto;" /> ] -- .pull-right[ <br/> * We do observe a gender gap for each education group * But it's not really clear if/how it varies with education * Let's test it with a regression! ] --- class:inverse # Task 4 (10 minutes) 1. Load the data `CPS1985` from the `AER` package. 1. Look at the `help` to get the definition of each variable: `?CPS1985` 1. We don't know if people are working part-time or full-time, does it matter here? 1. Create the`log_wage` variable equal to the log of `wage`. 1. Regress `log_wage` on `gender` and `education`, and save it as `reg1`. Interpret each coefficient. 1. Regress the `log_wage` on `gender`, `education` and their interaction `gender*education`, save it as `reg2`. Interpret each coefficient. Does the gender wage gap decrease with education? 1. Add all of other regressors to `reg2` and save it as `reg3`. Do our coefficients of interest change? --- # Wage, Gender and Education - Visually (Simple Model) <img src="chapter4_files/figure-html/unnamed-chunk-24-1.svg" style="display: block; margin: auto;" /> --- # Teaser for the Next 3 Lectures * You may have noticed that since the beginning we always work with **samples** drawn from the overall population. -- * Each time, imagine we could draw another sample from population: * Would we obtain the same results? * In other words, how confident can we be that our estimates (sign, magnitude) are not just driven by hazard? -- * The next chapters will answwer those kind of questions: * We'll present the notion of **sampling**, and * Understand what **statistical inference** is and how to do it. --- class: title-slide-final, middle background-image: url(../img/logo/ScPo-econ.png) background-size: 250px background-position: 9% 19% # SEE YOU NEXT WEEK! | | | | :--------------------------------------------------------------------------------------------------------- | :-------------------------------- | | <a href="mailto:michele.fioretti@sciencespo.fr">.ScPored[<i class="fa fa-paper-plane fa-fw"></i>] | michele.fioretti@sciencespo.fr | | <a href="https://michelefioretti.github.io/ScPoEconometrics-Slides/">.ScPored[<i class="fa fa-link fa-fw"></i>] | Slides | | <a href="https://michelefioretti.github.io/ScPoEconometrics/">.ScPored[<i class="fa fa-link fa-fw"></i>] | Book | | <a href="http://twitter.com/ScPoEcon">.ScPored[<i class="fa fa-twitter fa-fw"></i>] | @ScPoEcon | | <a href="http://github.com/ScPoEcon">.ScPored[<i class="fa fa-github fa-fw"></i>] | @ScPoEcon |