class: title-slide <br><br><br> # Lecture 3 ## Structural Modeling Workflow ### Tyler Ransom ### ECON 6343, University of Oklahoma --- # Today - What steps are required to estimate a structural model? - Go through each step on an example model --- # Steps we won't discuss today - The material we discuss today will already assume you have data - And that you have sufficient understanding of your data - It also assumes you have an understanding of your preferred coding language - These are all non-trivial steps, but they are typically covered in other classes - I will (indirectly) try to help you develop these skills throughout the course --- # Steps to performing structural estimation Mike Keane gave a [talk](https://www.youtube.com/watch?v=0hazaPBAYWE) at the University of Chicago in 2015 and listed these steps: 1. Theoretical Model Development 2. Practical Specification Issues 3. Solving the Model 4. Understanding How the Model Works 5. Estimation 6. Validation 7. Policy Experiments --- # An example model To help fix ideas, let's revisit a commonly used model in introductory econometrics: $$ `\begin{align} \log(w_{i}) =& \beta_0 + \beta_1 s_{i} + \beta_2 x_{i} + \beta_3 x^2_{i} + \varepsilon_{i} \label{eq:basicmincer} \end{align}` $$ where we have cross-sectional data and where - `\(i\)` indexes individuals - `\(w_{i}\)` is employment income - `\(s_{i}\)` is years of schooling - `\(x_{i}\)` is years of work experience (or, more commonly, _potential_ work experience) - `\(\varepsilon_{i}\)` is anything else that determines income We want to estimate `\(\left(\beta_1,\beta_2,\beta_3\right)\)`, which are .hi[returns to human capital investment] --- # Quick review - It is nearly certain that \eqref{eq:basicmincer} suffers from omitted variable bias - i.e. there are lots of factors in `\(\varepsilon_{i}\)` that are correlated with both `\(s_i\)` and `\(w_i\)` - Thus, our estimates of `\(\left(\beta_1,\beta_2,\beta_3\right)\)` will not be causal - We could try to get causal estimates using a variety of identification strategies: - find a valid instrument for `\(s_i\)` (Angrist and Krueger, 1991; Card, 1995) - exploit a discontinuity in `\(s_i\)` (Ost, Pan, and Webber, 2018) - randomize `\(s_i\)` (Attanasio, Meghir, and Santiago, 2011) - etc. --- # A structural view of Equation \eqref{eq:basicmincer} - We know that \eqref{eq:basicmincer} will produced biased estimates, but _why_? Some possibilities: - .hi[ability bias] - `\(s_i\)` and `\(w_i\)` are both positively correlated with unobservable cognitive ability - .hi[comparative advantage] - multidimensional unobservable ability `\(\implies\)` self-selection into schooling - .hi[credit constraints] - `\(s_i\)` is a costly investment; some people may not be able to borrow enough - .hi[preference heterogeneity] (differing tastes for `\(s_i\)`, differing discount rates) --- # 1. Theoretical Model Development - Since schooling has an up-front cost and long-term benefit, need a dynamic model - period 1: decide how much schooling to get - period 2: choose whether or not to work; if working, receive income by \eqref{eq:basicmincer} - individuals choose schooling level to maximize lifetime utility - Preferences (denote utility in period `\(t\)` by `\(u_t\)`, with `\(s,x\)` and `\(w\)` defined previously) $$ `\begin{align} u_1\left(z,c,\eta_1\right) & = f\left(z,c,\eta_1\right) \nonumber \\ u_2\left(w\left(s,x\right),k,\eta_2\right) & = g\left(w\left(s,x\right),k,\eta_2\right) \\ \label{eq:utils} \end{align}` $$ where `\(z\)` is family background, `\(c\)` is schooling costs, `\(k\)` is number of kids in adult household and `\(\eta_t\)` are unobservable preferences [similar to `\(\varepsilon\)` in \eqref{eq:basicmincer}] --- # 1. Theoretical Model Development With discount factor `\(\delta \in \left[0,1\right]\)`, the discounted lifetime utility function is then $$ `\begin{align} V & = u_1\left(z,c,\eta_1\right) + \delta u_2\left(w\left(s,x\right),k,\eta_2\right) \label{eq:PDV} \end{align}` $$ - Equations \eqref{eq:basicmincer}–\eqref{eq:PDV} define our model - This model is still .hi[laughably unrealistic], but at least we have something - A number of important questions arise (But we'll ignore these for today) - Where is cognitive ability? What exactly does `\(c\)` represent? Where are loans? - Maybe people should care about _consumption_ in period 2, not income - Does family background really only enter `\(u_1\)` and not `\(\log\left(w\right)\)`? - Should `\(x\)` in \eqref{eq:basicmincer} be a function of `\(s\)`? (Lower `\(s \implies\)` longer working life) - What are people's beliefs about future `\(k\)` when deciding `\(s\)`? --- # Overview of the theoretical model - As you can see, it takes a lot of know-how to write down even a simple model - Requires knowledge about the subject and about math/econ more generally .smallest[ .pull-left[ .hi[Exogenous variables] - family background `\((z)\)` - schooling costs `\((c)\)` - children in household `\((k)\)` .hi[Endogenous variables] - schooling `\((s)\)` - period-2 work decision ] .pull-right[ .hi[Outcome variable] - labor income `\((w)\)` .hi[Unobservables] - income `\((\varepsilon)\)` - preferences `\((\eta_t)\)` .hi[Model parameters] - returns to human capital `\((\beta)\)` - discount factor `\((\delta)\)` - other parameters implied by `\(f(\cdot)\)` and `\(g(\cdot)\)` ] ] --- # 2. Practical Specification Issues - Now that we have a model, we need to figure out how to take it to data - This is where we apply knowledge about .hi[our data] and .hi[stats/econometrics] - Key data questions: - Can we observe the variables of the model in our data set? - If so, are they reliably measured? - Key specification questions: - How to model `\(\eta_t\)` and `\(\varepsilon\)`? (Need to make distributional assumptions) - Functional forms of `\(f(\cdot)\)` and `\(g(\cdot)\)` - Should `\(s\)` be continuous (years of schooling) or discrete (college/not)? --- # 2. Practical Specification Issues - We won't get into too many details about this today, but specification is important! - What determines the specification is often: - what is reliably measured in the data - what is computationally feasible to estimate - Parameters of the model either need to be .hi[estimated] or .hi[calibrated] - e.g. often we don't have reliable data to allow us to estimate `\(\delta\)`; we must calibrate it - Computational feasibility often governs how we specify the different functions - e.g. _linear-in-parameters_ with _additively separable_ unobservables [like \eqref{eq:basicmincer}] --- # Example with real data - Here is some real data from the most recent round of the NLSY97 .scroll-box-12[ ``` julia using CSV, DataFrames, Statistics df = CSV.read("Data/slides3data.csv"; missingstrings=["NA"]) size(df) # outputs (6009, 12) describe(df) # outputs the below: 12×8 DataFrame │ Row │ variable │ mean │ min │ median │ max │ nunique │ nmissing │ │ │ Symbol │ Float64 │ Real │ Float64 │ Real │ Nothing │ Union… │ ├─────┼────────────────┼──────────┼──────┼─────────┼─────────┼─────────┼──────────┤ │ 1 │ id │ 4534.71 │ 4 │ 4544.0 │ 9022 │ │ │ │ 2 │ female │ 0.52671 │ 0 │ 1.0 │ 1 │ │ │ │ 3 │ black │ 0.269762 │ 0 │ 0.0 │ 1 │ │ │ │ 4 │ latin │ 0.210351 │ 0 │ 0.0 │ 1 │ │ │ │ 5 │ white │ 0.511067 │ 0 │ 1.0 │ 1 │ │ │ │ 6 │ employed │ 0.756532 │ 0 │ 1.0 │ 1 │ │ │ │ 7 │ wage │ 25.5309 │ 8.0 │ 20.0 │ 150.0 │ │ 933 │ │ 8 │ collgrad │ 0.350474 │ 0 │ 0.0 │ 1 │ │ │ │ 9 │ age │ 34.967 │ 33 │ 35.0 │ 37 │ │ │ │ 10 │ parent_college │ 0.238975 │ 0 │ 0.0 │ 1 │ │ │ │ 11 │ numkids │ 1.32684 │ 0 │ 1.0 │ 9 │ │ │ │ 12 │ efc │ 4.2243 │ 0.0 │ 0.77763 │ 118.111 │ │ │ ``` ] - We have demographics/background, wages, employment status, education, fertility - N=6009, age `\(\in \{33,\ldots,37\}\)`, and 35% of respondents graduated college - 24% have at least one college-graduate parent --- # Example: setting up the specification - It looks like we can estimate some form of our model - We have family background, cost of college (this is the `efc` variable) - We have employment status, wage and number of children - It looks like we'll have to have `\(s\)` be binary (`collgrad` variable) - Also need to assume `\(x = age - 18\)` if non-grad, `\(x = age - 22\)` if grad (Mincer, 1974) - Then we just need to add some functional form assumptions, and we'll be ready - `\(\varepsilon \sim\)` Normal, `\(\eta_t \sim\)` Logistic - `\(u_{i1} = \alpha_0 + \alpha_1 \text{ parent_college} + \alpha_2 \text{ efc} + \eta_1\)` - `\(u_{i2} = \gamma_0 + \gamma_1 \mathbb{E} \log w_{i} + \gamma_2 \text{ numkids} + \eta_2\)` --- # Parameters of the empirical model - We can now detail the parameters of the empirical model - .hi[wage parameters] `\((\beta,\sigma_{\varepsilon})\)` - The latter is the std. dev. of income shocks - .hi[schooling parameters] `\((\alpha)\)` - .hi[employment parameters] `\((\gamma,\delta)\)` - Then write down a statistical objective function as a fn. of data and parameters - e.g. maximize the likelihood, or minimize the sum of the squared residuals - We'll learn how to do this in later classes, but not today --- # 3. Solving and 4. Understanding How the Model Works - .hi[Solving the model:] - solve the dynamic utility max problem for given parameter values - (we aren't estimating parameter values yet) - (we will talk about how to do this next week) - .hi[Understanding the model:] - simulate data from the model - make sure the simulated data is consistent with the model's implications - look at descriptive statistics from the simulated data --- # 3. Solving and 4. Understanding How the Model Works - Start with as simple of a model as possible; make sure things are working - When introducing more complexities, do "numerical comparative statics" - Make sure the parameters move in the correct directions - e.g. `\(\uparrow \beta_1 \implies \uparrow\)` schooling (ceteris paribus) - If they don't, you've likely got a bug somewhere --- # Example with real data - How would we do this in Julia? - We can simulate log wages and then see how close we got - This is kind of silly in our simple model, but the workflow is there ``` julia N = size(df,1) β = [1.65,.4,.06,-.0002] σ = .4; df.exper = df.age .- ( 18*(1 .- df.collgrad) .+ 22*df.collgrad ) df.lwsim = β[1] .+ β[2]*df.collgrad .+ β[3]*df.exper .+ β[4]*df.exper.^2 .+ σ*randn(N) df.lw = log.(df.wage) ``` - We can then compare how `df.lwsim` compares with `df.lw` in the data .scroll-box-4[ ``` julia describe(df;cols=[:lw,:lwsim]) # returns │ Row │ variable │ mean │ min │ median │ max │ nunique │ nmissing │ eltype │ │ │ Symbol │ Float64 │ Float64 │ Float64 │ Float64 │ Nothing │ Union… │ Type │ ├─────┼──────────┼─────────┼─────────┼─────────┼─────────┼─────────┼──────────┼─────────────────────────┤ │ 1 │ lw │ 3.06219 │ 2.07944 │ 2.99573 │ 5.01064 │ │ 933 │ Union{Missing, Float64} │ │ 2 │ lwsim │ 2.67169 │ 1.12192 │ 2.67668 │ 3.98557 │ │ │ Float64 │ ``` ] --- # 5. Estimation - Most structural models require .hi[nonlinear estimation] - e.g. MLE/GMM or their simulated counterparts - In nonlinear optimization, starting values are crucial - Initializing at random starting values is likely to give poor results - Keane recommends calibrating the model by hand - e.g. match the intercept of each equation to the `\(\overline{Y}\)`'s in the data - I recommend estimating an intercepts-only model (or with very few `\(X\)`'s) - But this advice is model-specific! --- # 5. Estimation - There are lots of algorithms for nonlinear optimization - We'll talk more about these later in the course - Your next problem set will show how to do this in Julia --- # Example using real data - In our simple model, we can get good starting values by estimating OLS and logits - The wage equation can be estimated by OLS (on the subsample who are employed) ``` julia using GLM β̂ = lm(@formula(lw ~ collgrad + exper + exper^2), df[df.employed.==1,:]) # returns Coefficients: ───────────────────────────────────────────────────────────────────────────────── Estimate Std. Error t value Pr(>|t|) Lower 95% Upper 95% ───────────────────────────────────────────────────────────────────────────────── (Intercept) 2.94607 0.323145 9.11688 <1e-18 2.31255 3.57959 collgrad 0.534326 0.0271395 19.6881 <1e-82 0.481119 0.587532 exper -0.0265561 0.0412115 -0.644386 0.5194 -0.107351 0.0542385 exper ^ 2 0.0014304 0.00132307 1.08112 0.2797 -0.00116346 0.00402426 ───────────────────────────────────────────────────────────────────────────────── df.elwage = predict(β̂, df) # generates expected log wage for all observations r2(β̂) # reports R2 sqrt(deviance(β̂)/dof_residual(β̂)) # reports root mean squared error ``` --- # Example using real data - The `\(u_t\)` equations can be estimated as simple logits (on the full sample) .scroll-box-14[ ``` julia α̂ = glm(@formula(collgrad ~ parent_college + efc), df, Binomial(), LogitLink()) # returns Coefficients: ────────────────────────────────────────────────────────────────────────────────── Estimate Std. Error z value Pr(>|z|) Lower 95% Upper 95% ────────────────────────────────────────────────────────────────────────────────── (Intercept) -1.20091 0.0364888 -32.9118 <1e-99 -1.27243 -1.1294 parent_college 1.47866 0.068433 21.6074 <1e-99 1.34453 1.61278 efc 0.0450253 0.00437704 10.2867 <1e-24 0.0364464 0.0536041 ────────────────────────────────────────────────────────────────────────────────── γ̂ = glm(@formula(employed ~ elwage + numkids), df, Binomial(), LogitLink()) # returns Coefficients: ────────────────────────────────────────────────────────────────────────────── Estimate Std. Error z value Pr(>|z|) Lower 95% Upper 95% ────────────────────────────────────────────────────────────────────────────── (Intercept) -4.25036 0.454826 -9.34503 <1e-20 -5.1418 -3.35892 elwage 1.80081 0.149078 12.0796 <1e-32 1.50863 2.093 numkids -0.0797204 0.0218106 -3.65512 0.0003 -0.122468 -0.0369724 ────────────────────────────────────────────────────────────────────────────── ``` ] --- # Do these results make sense? - It can be informative to try and interpret even these simple results - wage equation: - insignificant return to experience is surprising; otherwise makes sense - schooling choice: - If `efc` captures college costs, it should have a negative sign - This suggests omitted variable bias in this equation - employment choice: - These results check out; may want to introduce nonlinearities in `numkids` --- # 6. Validation - If you have a good model, it should be .hi[valid] (i.e. predict well out of sample) - Validation is not always possible, but it's good to do if you can - e.g. if experimental data, estimate on control group, validate on treatment group - e.g. see if model can replicate major policy change in data - More simply, you could throw out half your data, then try to predict other half - This is typically not done if the full sample isn't huge --- # 7. Policy Experiments - This is the main reason to do structural estimation! - Structural estimation `\(\implies\)` recovering the DGP of the model - Once we know the DGP, we can simulate data from it and do policy experiments - requires having policy-invariant parameters! - We can predict the effects of: - proposed policies - hypothetical policies - Contrast with RCTs, which only reveal effects of implemented policies --- # Example using real data - We have two policy variables we could play with 1. `efc` (i.e. how much gov't subsidizes college tuition & fees) 2. return to schooling (this could change due to e.g. technological change) - Here's how we would look at a counterfactual with lower cost: .scroll-box-4[ ``` julia df_cfl = deepcopy(df) df_cfl.efc = df.efc .- 1 # change value of efc to be $1,000 less df.basesch = predict(α̂, df) # predicted collgrad probabilities under status quo df.cflsch = predict(α̂, df_cfl) # predicted collgrad probabilities under counterfactual describe(df;cols=[:basesch,:cflsch]) # returns │ Row │ variable │ mean │ min │ median │ max │ nunique │ nmissing │ │ │ Symbol │ Float64 │ Float64 │ Float64 │ Float64 │ Nothing │ Int64 │ ├─────┼──────────┼──────────┼──────────┼──────────┼──────────┼─────────┼──────────┤ │ 1 │ basesch │ 0.350474 │ 0.231313 │ 0.24387 │ 0.986715 │ │ 0 │ │ 2 │ cflsch │ 0.341794 │ 0.223404 │ 0.235663 │ 0.986111 │ │ 0 │ ``` ] - Average likelihood of `collgrad` _declines_ from 35% to 34.2% - This doesn't make sense because the `efc` coefficient didn't make sense --- # Example using real data - We can't assess the counterfactual of increasing the return to schooling - Because `elwage` doesn't directly enter the `collgrad` logit model - This is because we aren't really estimating the dynamic model yet - We will learn how to do this in the near future --- # In summary: Why structural estimation? - Want to examine effects of policies not yet implemented - Learn more about economics by looking through the lens of a model - Assess performance of theoretical models in explaining real-world data - Can be used to build up long-run "canonical" models of behavior in many areas - It can be really fun to do more complicated econometrics beyond simple regressions - Observational data is much cheaper to collect than experimental data --- # In summary: Why _not_ structural estimation? - It's really difficult to write down and estimate a tractable, realistic model! - It requires additional effort beyond data preparation and running regressions - Understanding identification of the model takes a lot of effort, too - It can be really miserable to try and debug the code of a structural estimation - Many structural models can take weeks to estimate one specification - in addition to months spent coding/debugging beforehand - As you can see, even with a simple model things have already gotten complicated! --- # References .smallest[ Ackerberg, D. A. (2003). "Advertising, Learning, and Consumer Choice in Experience Good Markets: An Empirical Examination". In: _International Economic Review_ 44.3, pp. 1007-1040. DOI: [10.1111/1468-2354.t01-2-00098](https://doi.org/10.1111%2F1468-2354.t01-2-00098). Adams, R. P. (2018). _Model Selection and Cross Validation_. Lecture Notes. Princeton University. URL: [https://www.cs.princeton.edu/courses/archive/fall18/cos324/files/model-selection.pdf](https://www.cs.princeton.edu/courses/archive/fall18/cos324/files/model-selection.pdf). Ahlfeldt, G. M., S. J. Redding, D. M. Sturm, et al. (2015). "The Economics of Density: Evidence From the Berlin Wall". In: _Econometrica_ 83.6, pp. 2127-2189. DOI: [10.3982/ECTA10876](https://doi.org/10.3982%2FECTA10876). Altonji, J. G., T. E. Elder, and C. R. Taber (2005). "Selection on Observed and Unobserved Variables: Assessing the Effectiveness of Catholic Schools". In: _Journal of Political Economy_ 113.1, pp. 151-184. DOI: [10.1086/426036](https://doi.org/10.1086%2F426036). Altonji, J. G. and C. R. Pierret (2001). "Employer Learning and Statistical Discrimination". In: _Quarterly Journal of Economics_ 116.1, pp. 313-350. DOI: [10.1162/003355301556329](https://doi.org/10.1162%2F003355301556329). Angrist, J. D. and A. B. Krueger (1991). "Does Compulsory School Attendance Affect Schooling and Earnings?" In: _Quarterly Journal of Economics_ 106.4, pp. 979-1014. DOI: [10.2307/2937954](https://doi.org/10.2307%2F2937954). Angrist, J. D. and J. Pischke (2009). _Mostly Harmless Econometrics: An Empiricist's Companion_. Princeton University Press. ISBN: 0691120358. Arcidiacono, P. (2004). "Ability Sorting and the Returns to College Major". In: _Journal of Econometrics_ 121, pp. 343-375. DOI: [10.1016/j.jeconom.2003.10.010](https://doi.org/10.1016%2Fj.jeconom.2003.10.010). Arcidiacono, P., E. Aucejo, A. Maurel, et al. (2016). _College Attrition and the Dynamics of Information Revelation_. Working Paper. Duke University. URL: [https://tyleransom.github.io/research/CollegeDropout2016May31.pdf](https://tyleransom.github.io/research/CollegeDropout2016May31.pdf). Arcidiacono, P., E. Aucejo, A. Maurel, et al. (2025). "College Attrition and the Dynamics of Information Revelation". In: _Journal of Political Economy_ 133.1. DOI: [10.1086/732526](https://doi.org/10.1086%2F732526). Arcidiacono, P. and J. B. Jones (2003). "Finite Mixture Distributions, Sequential Likelihood and the EM Algorithm". In: _Econometrica_ 71.3, pp. 933-946. DOI: [10.1111/1468-0262.00431](https://doi.org/10.1111%2F1468-0262.00431). Arcidiacono, P., J. Kinsler, and T. Ransom (2022b). "Asian American Discrimination in Harvard Admissions". In: _European Economic Review_ 144, p. 104079. DOI: [10.1016/j.euroecorev.2022.104079](https://doi.org/10.1016%2Fj.euroecorev.2022.104079). Arcidiacono, P., J. Kinsler, and T. Ransom (2022a). "Legacy and Athlete Preferences at Harvard". In: _Journal of Labor Economics_ 40.1, pp. 133-156. DOI: [10.1086/713744](https://doi.org/10.1086%2F713744). Arcidiacono, P. and R. A. Miller (2011). "Conditional Choice Probability Estimation of Dynamic Discrete Choice Models With Unobserved Heterogeneity". In: _Econometrica_ 79.6, pp. 1823-1867. DOI: [10.3982/ECTA7743](https://doi.org/10.3982%2FECTA7743). Arroyo Marioli, F., F. Bullano, S. Kucinskas, et al. (2020). _Tracking R of COVID-19: A New Real-Time Estimation Using the Kalman Filter_. Working Paper. medRxiv. DOI: [10.1101/2020.04.19.20071886](https://doi.org/10.1101%2F2020.04.19.20071886). Ashworth, J., V. J. Hotz, A. Maurel, et al. (2021). "Changes across Cohorts in Wage Returns to Schooling and Early Work Experiences". In: _Journal of Labor Economics_ 39.4, pp. 931-964. DOI: [10.1086/711851](https://doi.org/10.1086%2F711851). Attanasio, O. P., C. Meghir, and A. Santiago (2011). "Education Choices in Mexico: Using a Structural Model and a Randomized Experiment to Evaluate PROGRESA". In: _Review of Economic Studies_ 79.1, pp. 37-66. DOI: [10.1093/restud/rdr015](https://doi.org/10.1093%2Frestud%2Frdr015). Aucejo, E. M. and J. James (2019). "Catching Up to Girls: Understanding the Gender Imbalance in Educational Attainment Within Race". In: _Journal of Applied Econometrics_ 34.4, pp. 502-525. DOI: [10.1002/jae.2699](https://doi.org/10.1002%2Fjae.2699). Baragatti, M., A. Grimaud, and D. Pommeret (2013). "Likelihood-free Parallel Tempering". In: _Statistics and Computing_ 23.4, pp. 535-549. DOI: [ 10.1007/s11222-012-9328-6](https://doi.org/%2010.1007%2Fs11222-012-9328-6). Bayer, P., R. McMillan, A. Murphy, et al. (2016). "A Dynamic Model of Demand for Houses and Neighborhoods". In: _Econometrica_ 84.3, pp. 893-942. DOI: [10.3982/ECTA10170](https://doi.org/10.3982%2FECTA10170). Begg, C. B. and R. Gray (1984). "Calculation of Polychotomous Logistic Regression Parameters Using Individualized Regressions". In: _Biometrika_ 71.1, pp. 11-18. DOI: [10.1093/biomet/71.1.11](https://doi.org/10.1093%2Fbiomet%2F71.1.11). Beggs, S. D., N. S. Cardell, and J. Hausman (1981). "Assessing the Potential Demand for Electric Cars". In: _Journal of Econometrics_ 17.1, pp. 1-19. DOI: [10.1016/0304-4076(81)90056-7](https://doi.org/10.1016%2F0304-4076%2881%2990056-7). Berry, S., J. Levinsohn, and A. Pakes (1995). "Automobile Prices in Market Equilibrium". In: _Econometrica_ 63.4, pp. 841-890. URL: [http://www.jstor.org/stable/2171802](http://www.jstor.org/stable/2171802). Blass, A. A., S. Lach, and C. F. Manski (2010). "Using Elicited Choice Probabilities to Estimate Random Utility Models: Preferences for Electricity Reliability". In: _International Economic Review_ 51.2, pp. 421-440. DOI: [10.1111/j.1468-2354.2010.00586.x](https://doi.org/10.1111%2Fj.1468-2354.2010.00586.x). Blundell, R. (2010). "Comments on: ``Structural vs. Atheoretic Approaches to Econometrics'' by Michael Keane". In: _Journal of Econometrics_ 156.1, pp. 25-26. DOI: [10.1016/j.jeconom.2009.09.005](https://doi.org/10.1016%2Fj.jeconom.2009.09.005). Bresnahan, T. F., S. Stern, and M. Trajtenberg (1997). "Market Segmentation and the Sources of Rents from Innovation: Personal Computers in the Late 1980s". In: _The RAND Journal of Economics_ 28.0, pp. S17-S44. DOI: [10.2307/3087454](https://doi.org/10.2307%2F3087454). Brien, M. J., L. A. Lillard, and S. Stern (2006). "Cohabitation, Marriage, and Divorce in a Model of Match Quality". In: _International Economic Review_ 47.2, pp. 451-494. DOI: [10.1111/j.1468-2354.2006.00385.x](https://doi.org/10.1111%2Fj.1468-2354.2006.00385.x). Card, D. (1995). "Using Geographic Variation in College Proximity to Estimate the Return to Schooling". In: _Aspects of Labor Market Behaviour: Essays in Honour of John Vanderkamp_. Ed. by L. N. Christofides, E. K. Grant and R. Swidinsky. Toronto: University of Toronto Press. Cardell, N. S. (1997). "Variance Components Structures for the Extreme-Value and Logistic Distributions with Application to Models of Heterogeneity". In: _Econometric Theory_ 13.2, pp. 185-213. URL: [https://www.jstor.org/stable/3532724](https://www.jstor.org/stable/3532724). Caucutt, E. M., L. Lochner, J. Mullins, et al. (2020). _Child Skill Production: Accounting for Parental and Market-Based Time and Goods Investments_. Working Paper 27838. National Bureau of Economic Research. DOI: [10.3386/w27838](https://doi.org/10.3386%2Fw27838). Chen, X., H. Hong, and D. Nekipelov (2011). "Nonlinear Models of Measurement Errors". In: _Journal of Economic Literature_ 49.4, pp. 901-937. DOI: [10.1257/jel.49.4.901](https://doi.org/10.1257%2Fjel.49.4.901). Chintagunta, P. K. (1992). "Estimating a Multinomial Probit Model of Brand Choice Using the Method of Simulated Moments". In: _Marketing Science_ 11.4, pp. 386-407. DOI: [10.1287/mksc.11.4.386](https://doi.org/10.1287%2Fmksc.11.4.386). Cinelli, C. and C. Hazlett (2020). "Making Sense of Sensitivity: Extending Omitted Variable Bias". In: _Journal of the Royal Statistical Society: Series B (Statistical Methodology)_ 82.1, pp. 39-67. DOI: [10.1111/rssb.12348](https://doi.org/10.1111%2Frssb.12348). Coate, P. and K. Mangum (2019). _Fast Locations and Slowing Labor Mobility_. Working Paper 19-49. Federal Reserve Bank of Philadelphia. Cunha, F., J. J. Heckman, and S. M. Schennach (2010). "Estimating the Technology of Cognitive and Noncognitive Skill Formation". In: _Econometrica_ 78.3, pp. 883-931. DOI: [10.3982/ECTA6551](https://doi.org/10.3982%2FECTA6551). Cunningham, S. (2021). _Causal Inference: The Mixtape_. Yale University Press. URL: [https://www.scunning.com/causalinference_norap.pdf](https://www.scunning.com/causalinference_norap.pdf). Delavande, A. and C. F. Manski (2015). "Using Elicited Choice Probabilities in Hypothetical Elections to Study Decisions to Vote". In: _Electoral Studies_ 38, pp. 28-37. DOI: [10.1016/j.electstud.2015.01.006](https://doi.org/10.1016%2Fj.electstud.2015.01.006). Delavande, A. and B. Zafar (2019). "University Choice: The Role of Expected Earnings, Nonpecuniary Outcomes, and Financial Constraints". In: _Journal of Political Economy_ 127.5, pp. 2343-2393. DOI: [10.1086/701808](https://doi.org/10.1086%2F701808). Diegert, P., M. A. Masten, and A. Poirier (2025). _Assessing Omitted Variable Bias when the Controls are Endogenous_. arXiv. DOI: [10.48550/ARXIV.2206.02303](https://doi.org/10.48550%2FARXIV.2206.02303). Erdem, T. and M. P. Keane (1996). "Decision-Making under Uncertainty: Capturing Dynamic Brand Choice Processes in Turbulent Consumer Goods Markets". In: _Marketing Science_ 15.1, pp. 1-20. DOI: [10.1287/mksc.15.1.1](https://doi.org/10.1287%2Fmksc.15.1.1). Evans, R. W. (2018). _Simulated Method of Moments (SMM) Estimation_. QuantEcon Note. University of Chicago. URL: [https://notes.quantecon.org/submission/5b3db2ceb9eab00015b89f93](https://notes.quantecon.org/submission/5b3db2ceb9eab00015b89f93). Farber, H. S. and R. Gibbons (1996). "Learning and Wage Dynamics". In: _Quarterly Journal of Economics_ 111.4, pp. 1007-1047. DOI: [10.2307/2946706](https://doi.org/10.2307%2F2946706). Fu, C., N. Grau, and J. Rivera (2020). _Wandering Astray: Teenagers' Choices of Schooling and Crime_. Working Paper. University of Wisconsin-Madison. URL: [https://www.ssc.wisc.edu/~cfu/wander.pdf](https://www.ssc.wisc.edu/~cfu/wander.pdf). Gillingham, K., F. Iskhakov, A. Munk-Nielsen, et al. (2022). "Equilibrium Trade in Automobiles". In: _Journal of Political Economy_. DOI: [10.1086/720463](https://doi.org/10.1086%2F720463). Haile, P. (2019). _``Structural vs. Reduced Form'' Language and Models in Empirical Economics_. Lecture Slides. Yale University. URL: [http://www.econ.yale.edu/~pah29/intro.pdf](http://www.econ.yale.edu/~pah29/intro.pdf). Haile, P. (2024). _Models, Measurement, and the Language of Empirical Economics_. Lecture Slides. Yale University. URL: [https://www.dropbox.com/s/8kwtwn30dyac18s/intro.pdf](https://www.dropbox.com/s/8kwtwn30dyac18s/intro.pdf). Heckman, J. J., J. Stixrud, and S. Urzua (2006). "The Effects of Cognitive and Noncognitive Abilities on Labor Market Outcomes and Social Behavior". In: _Journal of Labor Economics_ 24.3, pp. 411-482. DOI: [10.1086/504455](https://doi.org/10.1086%2F504455). Hotz, V. J. and R. A. Miller (1993). "Conditional Choice Probabilities and the Estimation of Dynamic Models". In: _The Review of Economic Studies_ 60.3, pp. 497-529. DOI: [10.2307/2298122](https://doi.org/10.2307%2F2298122). Hurwicz, L. (1950). "Generalization of the Concept of Identification". In: _Statistical Inference in Dynamic Economic Models_. Hoboken, NJ: John Wiley and Sons, pp. 245-257. Ishimaru, S. (2022). _Geographic Mobility of Youth and Spatial Gaps in Local College and Labor Market Opportunities_. Working Paper. Hitotsubashi University. James, J. (2011). _Ability Matching and Occupational Choice_. Working Paper 11-25. Federal Reserve Bank of Cleveland. James, J. (2017). "MM Algorithm for General Mixed Multinomial Logit Models". In: _Journal of Applied Econometrics_ 32.4, pp. 841-857. DOI: [10.1002/jae.2532](https://doi.org/10.1002%2Fjae.2532). Jin, H. and H. Shen (2020). "Foreign Asset Accumulation Among Emerging Market Economies: A Case for Coordination". In: _Review of Economic Dynamics_ 35.1, pp. 54-73. DOI: [10.1016/j.red.2019.04.006](https://doi.org/10.1016%2Fj.red.2019.04.006). Keane, M. P. (2010). "Structural vs. Atheoretic Approaches to Econometrics". In: _Journal of Econometrics_ 156.1, pp. 3-20. DOI: [10.1016/j.jeconom.2009.09.003](https://doi.org/10.1016%2Fj.jeconom.2009.09.003). Keane, M. P. and K. I. Wolpin (1997). "The Career Decisions of Young Men". In: _Journal of Political Economy_ 105.3, pp. 473-522. DOI: [10.1086/262080](https://doi.org/10.1086%2F262080). Koopmans, T. C. and O. Reiersol (1950). "The Identification of Structural Characteristics". In: _The Annals of Mathematical Statistics_ 21.2, pp. 165-181. URL: [http://www.jstor.org/stable/2236899](http://www.jstor.org/stable/2236899). Kosar, G., T. Ransom, and W. van der Klaauw (2022). "Understanding Migration Aversion Using Elicited Counterfactual Choice Probabilities". In: _Journal of Econometrics_ 231.1, pp. 123-147. DOI: [10.1016/j.jeconom.2020.07.056](https://doi.org/10.1016%2Fj.jeconom.2020.07.056). Krauth, B. (2016). "Bounding a Linear Causal Effect Using Relative Correlation Restrictions". In: _Journal of Econometric Methods_ 5.1, pp. 117-141. DOI: [10.1515/jem-2013-0013](https://doi.org/10.1515%2Fjem-2013-0013). Lang, K. and M. D. Palacios (2018). _The Determinants of Teachers' Occupational Choice_. Working Paper 24883. National Bureau of Economic Research. DOI: [10.3386/w24883](https://doi.org/10.3386%2Fw24883). Lee, D. S., J. McCrary, M. J. Moreira, et al. (2020). _Valid t-ratio Inference for IV_. Working Paper. arXiv. URL: [https://arxiv.org/abs/2010.05058](https://arxiv.org/abs/2010.05058). Lewbel, A. (2019). "The Identification Zoo: Meanings of Identification in Econometrics". In: _Journal of Economic Literature_ 57.4, pp. 835-903. DOI: [10.1257/jel.20181361](https://doi.org/10.1257%2Fjel.20181361). Mahoney, N. (2022). "Principles for Combining Descriptive and Model-Based Analysis in Applied Microeconomics Research". In: _Journal of Economic Perspectives_ 36.3, pp. 211-22. DOI: [10.1257/jep.36.3.211](https://doi.org/10.1257%2Fjep.36.3.211). McFadden, D. (1978). "Modelling the Choice of Residential Location". In: _Spatial Interaction Theory and Planning Models_. Ed. by A. Karlqvist, L. Lundqvist, F. Snickers and J. W. Weibull. Amsterdam: North Holland, pp. 75-96. McFadden, D. (1989). "A Method of Simulated Moments for Estimation of Discrete Response Models Without Numerical Integration". In: _Econometrica_ 57.5, pp. 995-1026. DOI: [10.2307/1913621](https://doi.org/10.2307%2F1913621). URL: [http://www.jstor.org/stable/1913621](http://www.jstor.org/stable/1913621). Mellon, J. (2020). _Rain, Rain, Go Away: 137 Potential Exclusion-Restriction Violations for Studies Using Weather as an Instrumental Variable_. Working Paper. University of Manchester. URL: [https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3715610](https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3715610). Miller, R. A. (1984). "Job Matching and Occupational Choice". In: _Journal of Political Economy_ 92.6, pp. 1086-1120. DOI: [10.1086/261276](https://doi.org/10.1086%2F261276). Mincer, J. (1974). _Schooling, Experience and Earnings_. New York: Columbia University Press for National Bureau of Economic Research. Ost, B., W. Pan, and D. Webber (2018). "The Returns to College Persistence for Marginal Students: Regression Discontinuity Evidence from University Dismissal Policies". In: _Journal of Labor Economics_ 36.3, pp. 779-805. DOI: [10.1086/696204](https://doi.org/10.1086%2F696204). Oster, E. (2019). "Unobservable Selection and Coefficient Stability: Theory and Evidence". In: _Journal of Business & Economic Statistics_ 37.2, pp. 187-204. DOI: [10.1080/07350015.2016.1227711](https://doi.org/10.1080%2F07350015.2016.1227711). Pischke, S. (2007). _Lecture Notes on Measurement Error_. Lecture Notes. London School of Economics. URL: [http://econ.lse.ac.uk/staff/spischke/ec524/Merr_new.pdf](http://econ.lse.ac.uk/staff/spischke/ec524/Merr_new.pdf). Ransom, M. R. and T. Ransom (2018). "Do High School Sports Build or Reveal Character? Bounding Causal Estimates of Sports Participation". In: _Economics of Education Review_ 64, pp. 75-89. DOI: [10.1016/j.econedurev.2018.04.002](https://doi.org/10.1016%2Fj.econedurev.2018.04.002). Ransom, T. (2022). "Labor Market Frictions and Moving Costs of the Employed and Unemployed". In: _Journal of Human Resources_ 57.S, pp. S137-S166. DOI: [10.3368/jhr.monopsony.0219-10013R2](https://doi.org/10.3368%2Fjhr.monopsony.0219-10013R2). Rudik, I. (2020). "Optimal Climate Policy When Damages Are Unknown". In: _American Economic Journal: Economic Policy_ 12.2, pp. 340-373. DOI: [10.1257/pol.20160541](https://doi.org/10.1257%2Fpol.20160541). Rust, J. (1987). "Optimal Replacement of GMC Bus Engines: An Empirical Model of Harold Zurcher". In: _Econometrica_ 55.5, pp. 999-1033. URL: [http://www.jstor.org/stable/1911259](http://www.jstor.org/stable/1911259). Shalizi, C. R. (2019). _Advanced Data Analysis from an Elementary Point of View_. Cambridge University Press. URL: [http://www.stat.cmu.edu/~cshalizi/ADAfaEPoV/ADAfaEPoV.pdf](http://www.stat.cmu.edu/~cshalizi/ADAfaEPoV/ADAfaEPoV.pdf). Smith Jr., A. A. (2008). "Indirect Inference". In: _The New Palgrave Dictionary of Economics_. Ed. by S. N. Durlauf and L. E. Blume. Vol. 1-8. London: Palgrave Macmillan. DOI: [10.1007/978-1-349-58802-2](https://doi.org/10.1007%2F978-1-349-58802-2). URL: [http://www.econ.yale.edu/smith/palgrave7.pdf](http://www.econ.yale.edu/smith/palgrave7.pdf). Stinebrickner, R. and T. Stinebrickner (2014a). "Academic Performance and College Dropout: Using Longitudinal Expectations Data to Estimate a Learning Model". In: _Journal of Labor Economics_ 32.3, pp. 601-644. DOI: [10.1086/675308](https://doi.org/10.1086%2F675308). Stinebrickner, R. and T. R. Stinebrickner (2014b). "A Major in Science? Initial Beliefs and Final Outcomes for College Major and Dropout". In: _Review of Economic Studies_ 81.1, pp. 426-472. DOI: [10.1093/restud/rdt025](https://doi.org/10.1093%2Frestud%2Frdt025). Su, C. and K. L. Judd (2012). "Constrained Optimization Approaches to Estimation of Structural Models". In: _Econometrica_ 80.5, pp. 2213-2230. DOI: [10.3982/ECTA7925](https://doi.org/10.3982%2FECTA7925). Train, K. (2009). _Discrete Choice Methods with Simulation_. 2nd ed. Cambridge; New York: Cambridge University Press. ISBN: 9780521766555. Wiswall, M. and B. Zafar (2018). "Preference for the Workplace, Investment in Human Capital, and Gender". In: _Quarterly Journal of Economics_ 133.1, pp. 457-507. DOI: [10.1093/qje/qjx035](https://doi.org/10.1093%2Fqje%2Fqjx035). Young, A. (2020). _Consistency without Inference: Instrumental Variables in Practical Application_. Working Paper. London School of Economics. ]