Lecture 11

class: title-slide

# Lecture 11

## Subjective Expectations and Stated Preferences

### Tyler Ransom

### ECON 6343, University of Oklahoma

---

# Plan for the Day

1. Discuss stated preferences

2. Estimating models using stated preference data

3. Discrete choice experiments

4. Stated probability experiments

---
# Stated preferences (SP) vs. Revealed preferences (RP)

- Up to now, we have been dealing with _revealed preference_ data

- The data we use includes choices that were _actually_ made

- In contrast, we can collect data on .hi[stated preferences]

- This data would include choices that are .hi[hypothetically] made

- See also: "If I had a million dollars" by the band Barenaked Ladies

---
# What are stated preferences?

- SP are derived from hypothetical situations

- e.g. "which of these three trucks would you purchase?"

- I've never purchased a truck before, so I wouldn't know which to choose

- But if I _were_ to purchase a truck, I suppose I'd pick a Toyota Tundra

- SP are tightly related to the counterfactuals we consider

---
# Strengths of SP and weaknesses of RP

.pull-left[
.center[.hi[RP]]
- usually don't know the choice set
- usually unable to observe important variables
- might not have enough variation in important dimension
- limited to settings that currently exist or have existed in the past
- have to worry about equilibrium allocation mechanism
- counterfactuals must be carefully undertaken
]

.pull-right[
.center[.hi[SP]]
- specify the complete choice set
- specify (and observe) all relevant variables
- govern how much variation is in the data
- we can obtain preferences for as-yet unseen settings
- can separate preferences from other parts of system
- individuals report `$y$` across many counterfactuals
]

---
# Cons of SP

- They are hypothetical! As such, they may not be grounded in reality

- If I had $1m, would I really buy a(nother) house? How much would I save?

- Data collection can't necessarily be used for multiple purposes

- May requires mastery of survey methodology

---
# Collection of RP data

- RP data is typically collected by governments

- e.g. household surveys, administrative data, etc.

- These data sources typically have many, many uses

- NLSY has been used in .hi[thousands] of papers

- Administrative data can be used for tax collection and economic research

- These data sources are widely available and comparable across time and other contexts

---
# Collection of SP data

- SP data are collected under a much narrower scope

- Individuals answer a series of hypothetical choice scenarios

- The `$X$`'s in each scenario are randomized

- Usually, the data can only be used for a limited number of research questions

- Because of this, researchers typically collect their own data

- This requires knowledge of best practices in survey methodology / experiments

- It also requires money to fund survey participation

- More work on the front end (survey development) but much easier to analyze

---
# Example hypothetical choice scenarios

What is the percent chance you would choose to live in each of these three locations given their characteristics below? .hi[Assume that the locations are otherwise identical.]

.hi[Scenario 1]

Option | Distance from current location | Family here | Income | Probability
--------|---------|---------|---------|---------
A (not move) | 0          | No  | 30% lower  | 
B            | 1000 miles | Yes | same       | 
C            | 1000 miles | No  | 30% higher |

.hi[Scenario 2]

Option | Distance from current location | Family here | Income | Probability
--------|---------|---------|---------|---------
A (not move) | 0          | Yes | 30% lower   | 
B            | 500 miles  | Yes | 150% higher | 
C            | 100 miles  | No  | 60% higher  |

---
# Guidelines for choice experiments

- Try to keep things as short as possible (respondent fatigue)

- Make sure no option in any scenario is strictly dominated

- Number of options should not exceed 4

- Vary a small number of `$X$`'s at a time so as not to overload respondents
    - but make sure respondents know to hold all else equal!

- Need sufficient variation in the `$X$`'s of interest

- Scenarios should match up with real life as much as possible

- Should include a status-quo option if possible

---
# Types of SP choice experiments

- Discrete choice
    - Individuals select which of the `$J$` options they prefer
    - i.e. if `$J=3$` then the `$y$` vector will be `$[0,1,0]$`, `$[1,0,0]$`, or `$[0,0,1]$`

- Rank-ordered choice
    - Individuals provide their preference ordering of the `$J$` options

- Probabilistic choice
    - Individuals provide choice probabilities of each of the `$J$` options
    
- Each of these settings provides increasing amounts of information

---
# Estimating discrete choice models

- Estimation proceeds as if one has panel data on choices (Rust, 1987)

- Each choice scenario is another observation in the individual's panel

- Can estimate assuming multinomial logit, nested logit, mixed logit, etc.

---
# Estimating rank-ordered choice models

- Here, we use the .hi[exploded logit] model (Beggs, Cardell, and Hausman, 1981)

- The "choice probability" is the joint event of a particular ranking of options

- It's a product of logit `$P$`'s, where the choice set decreases as options are ranked:

`\begin{align*}
\Pr\left(\text{Ranking}=1,\ldots,J\right) &= \frac{\exp\left(Z_{i1}\gamma\right)}{\sum_{k=1}^J\exp\left(Z_{ik}\gamma\right)}\frac{\exp\left(Z_{i2}\gamma\right)}{\sum_{k=2}^J\exp\left(Z_{ik}\gamma\right)}\cdots\frac{\exp\left(Z_{iJ-1}\gamma\right)}{\sum_{k=J-1}^J\exp\left(Z_{ik}\gamma\right)}
\end{align*}`

- We can also add mixing to these probabilities to get a mixed exploded logit

- Note that rank ordering provides more information than 0/1 choice data

- We now know the relative preference of the `$J-1$` non-chosen options

---
# Stated probabilistic choice models

- Now, we observe a probability of choosing each alternative

- What does this information represent? It represents the person's uncertainty

- Specifically, it represents .hi[resolvable uncertainty:] uncertainty about unspecified attributes or states of the world in which choices ultimately will be made

- e.g. Q. "Will you go for Mexican or Thai food on Friday night?"
    - A. "Well, I'm not sure what mood I'll be in that day, but probably Mexican"
    - Q. "What do you mean by 'probably'?" 
    - A. "A 78% chance"

- If the person thinks there is no such uncertainty, then they can report `$p=0$` or `$p=1$`

---
# Estimating probabilistic choice models

- How do we proceed with estimation when `$y$` is itself a probability (not 0/1)?

- We invert the logit formula

- Consider the binomial logit as an example

`\begin{align*}
P_{i1} &= \frac{\exp\left(\left(Z_{i1}-Z_{i2}\right)\gamma\right)}{1+\exp\left(\left(Z_{i1}-Z_{i2}\right)\gamma\right)}
\end{align*}`

- After some algebra, we get

`\begin{align*}
\ln\left(\frac{P_{i1}}{1-P_{i1}}\right) &= \left(Z_{i1}-Z_{i2}\right)\gamma
\end{align*}`

- Now we're in a world where we can use OLS to estimate `$\gamma$`!

---
# Increasing amounts of information

- Stated probabilities provide more information than discrete choice or rank ordering

- Consider the following responses for an individual

Option | Discrete Choice | Rank Ordering | Stated Probability 1 | Stated Probability 2
--------|---------|---------|---------|---------
1 | 0 | 2 | 1   | 49
2 | 1 | 1 | 99  | 51
3 | 0 | 3 | 0   | 0

- The preference ordering is the same in each column

- But the implied preference intensity is much different in the last two columns

- A (0,1,0) discrete choice response corresponds to a (0,100,0) probability response

---
# Measurement error

- One concern with stated probabilities is measurement error

- Rather than report 99.5%, someone may just write 100%

- But if `$p=0$` or `$p=1$`, `$\ln\left(\frac{P_{i1}}{1-P_{i1}}\right)$` is undefined!

- In this case, we have to recode 0s or 1s to be small values (e.g. .001, .999)

- Then, to avoid cheating, we need to use LAD instead of OLS. New equation:

`\begin{align*}
\ln\left(\frac{\tilde{P}_{i1}}{1-\tilde{P}_{i1}}\right) &= \left(Z_{i1}-Z_{i2}\right)\gamma + \eta_{i1}
\end{align*}`
where `$\tilde{P}$` is the recoded probability and `$\eta_{i1}$` is the difference in measurement errors

---
# Estimation with `$J>2$`

- The above is the equation for a 2-option choice set

- With more than 2 options, we have more observations per scenario

- If `$J=3$`, we have

`\begin{align*}
\ln\left(\frac{\tilde{P}_{i1}}{\tilde{P}_{i2}}\right) &= \left(Z_{i1}-Z_{i2}\right)\gamma + \eta_{i1}\\
\ln\left(\frac{\tilde{P}_{i3}}{\tilde{P}_{i2}}\right) &= \left(Z_{i3}-Z_{i2}\right)\gamma + \eta_{i3}
\end{align*}`

- If we see `$N$` people each making `$T$` choices, then our data has `$NT(J-1)$` rows

---
# Preference Heterogeneity

- Major advantage of elicited probabilities: handling unobserved heterogeneity

- We can say more because we have more information about preferences

- Stated probabilities contain more information than 0/1 or rank ordering

- Rather than assuming a normal distribution in a mixed logit,

- We can instead trace out the mixing distribution nonparametrically

- We simply estimate individual-specific `$\gamma$`'s

- Resulting distribution is almost never normal (i.e. it's skewed, etc.)

---
# Example from Kosar et al. (2022)

.center[![pref-het](histo_wtp_family.jpg)]

- The WTP distribution is highly skewed
- Some people _really_ like living close to family
- Some people prefer to be apart from family (negative WTP)

---
# Follow-on studies to link to RP data

- Stated probabilistic choice experiments are not a silver bullet

- You're still reduced to the SP vs. RP conundrum

- Will people actually do what they told you they would do?

- To resolve this, most studies have to conduct follow-on surveys

- Rationale: in between survey waves, people make actual choices

- You can then see how well their SP compares to their RP

- I've never seen SP diverge from RP, but that could be due to publication bias

---
# Papers that use stated probability experiments

- Blass, Lach, and Manski (2010)

- Delavande and Manski (2015)

- Wiswall and Zafar (2018)

- Delavande and Zafar (2019)

- Kosar, Ransom, and van der Klaauw (2022)

- Many others

---
# How to estimate probabilstic choice models in Julia

- We can read in data from Kosar, Ransom, and van der Klaauw (2022)

- Each pair of adjacent rows is a choice scenario

- `ratio` is the log of ratio of the probabilities

.scroll-box-12[

``` julia
using DataFrames, HTTP, CSV, GLM, QuantileRegressions

url = "https://raw.githubusercontent.com/OU-PhD-Econometrics/fall-2021/master/LectureNotes/11-SubjExp/SCEmobilityExample.csv"
df = CSV.read(HTTP.get(url).body)

println(head(df))
│ Row │ scuid  │ ratio    │ dist    │ crime     │ income     │ moved │ scennum │ altnum │ wave  │ mvcost │ family │ norms │ homecost │ size    │ taxes │ schqual │ withincitymove │ copyhome │ blkscen │
│     │ Int64  │ Float64  │ Float64 │ Float64   │ Float64    │ Int64 │ Int64   │ Int64  │ Int64 │ Int64  │ Int64  │ Int64 │ Float64  │ Float64 │ Int64 │ Int64   │ Int64          │ Int64    │ Int64   │
├─────┼────────┼──────────┼─────────┼───────────┼────────────┼───────┼─────────┼────────┼───────┼────────┼────────┼───────┼──────────┼─────────┼───────┼─────────┼────────────────┼──────────┼─────────┤
│ 1   │ 119007 │ 6.85646  │ -5.0    │ -0.693147 │ -0.182322  │ -1    │ 1       │ 1      │ 1     │ 0      │ 0      │ 0     │ 0.0      │ 0.0     │ 0     │ 0       │ 0              │ 0        │ 25      │
│ 2   │ 119007 │ 3.91202  │ 5.0     │ -1.38629  │ -0.0870114 │ 0     │ 1       │ 3      │ 1     │ 0      │ 0      │ 0     │ 0.0      │ 0.0     │ 0     │ 0       │ 0              │ 0        │ 25      │
│ 3   │ 119007 │ 6.85646  │ -5.0    │ 0.0       │ -0.0487902 │ -1    │ 2       │ 1      │ 1     │ 0      │ 0      │ 0     │ 0.0      │ 0.0     │ 0     │ 0       │ 0              │ 0        │ 26      │
│ 4   │ 119007 │ 3.91202  │ 0.0     │ -0.693147 │ -0.0487902 │ 0     │ 2       │ 3      │ 1     │ 0      │ 0      │ 0     │ 0.0      │ 0.0     │ 0     │ 0       │ 0              │ 0        │ 26      │
│ 5   │ 119007 │ 2.94444  │ -5.0    │ 0.693147  │ -0.0571584 │ -1    │ 3       │ 1      │ 1     │ 0      │ 0      │ 0     │ 0.0      │ 0.0     │ 0     │ 0       │ 0              │ 0        │ 27      │
│ 6   │ 119007 │ -3.91202 │ 5.0     │ 1.38629   │ 0.154151   │ 0     │ 3       │ 3      │ 1     │ 0      │ 0      │ 0     │ 0.0      │ 0.0     │ 0     │ 0       │ 0              │ 0        │ 27      │

estimates = qreg(@formula(ratio ~ income + crime + dist + mvcost + family + norms + homecost + size + taxes + schqual + withincitymove + copyhome + moved), df, .5)
Coefficients:
────────────────────────────────────────────────────────────
                Quantile    Estimate   Std.Error     t value
────────────────────────────────────────────────────────────
(Intercept)          0.5  -0.0338533  0.0133281     -2.53999
income               0.5   3.75781    0.0364599    103.067
crime                0.5  -0.640542   0.020861     -30.7053
dist                 0.5  -0.0549714  0.00358389   -15.3385
mvcost               0.5  -0.0397001  0.001884     -21.0722
family               0.5   2.13173    0.0259934     82.0105
norms                0.5   0.154355   0.0155282      9.94031
homecost             0.5  -0.798607   0.0800967     -9.97054
size                 0.5   0.635994   0.043953      14.4699
taxes                0.5  -0.0403194  0.00424806    -9.49124
schqual              0.5   0.238732   0.0167284     14.271
withincitymove       0.5   1.69741    0.0287623     59.0151
copyhome             0.5   0.110198   0.0288403      3.82097
moved                0.5  -2.57907    0.0253557   -101.715
────────────────────────────────────────────────────────────
```
]

---
# Other details for estimation

- You need to bootstrap to get appropriate standard errors

- This is because you have panel data

- Cluster (at individual level) robust inference won't be enough

- To get individual-specific preference estimates, `qreg` at individual level

---
# References
.tiny[
Ackerberg, D. A. (2003). "Advertising, Learning, and Consumer Choice in Experience Good
Markets: An Empirical Examination". In: _International Economic Review_ 44.3, pp.
1007-1040. DOI:
[10.1111/1468-2354.t01-2-00098](https://doi.org/10.1111%2F1468-2354.t01-2-00098).

Adams, R. P. (2018). _Model Selection and Cross Validation_. Lecture Notes. Princeton
University. URL:
[https://www.cs.princeton.edu/courses/archive/fall18/cos324/files/model-selection.pdf](https://www.cs.princeton.edu/courses/archive/fall18/cos324/files/model-selection.pdf).

Ahlfeldt, G. M., S. J. Redding, D. M. Sturm, et al. (2015). "The Economics of Density:
Evidence From the Berlin Wall". In: _Econometrica_ 83.6, pp. 2127-2189. DOI:
[10.3982/ECTA10876](https://doi.org/10.3982%2FECTA10876).

Altonji, J. G., T. E. Elder, and C. R. Taber (2005). "Selection on Observed and Unobserved
Variables: Assessing the Effectiveness of Catholic Schools". In: _Journal of Political
Economy_ 113.1, pp. 151-184. DOI: [10.1086/426036](https://doi.org/10.1086%2F426036).

Altonji, J. G. and C. R. Pierret (2001). "Employer Learning and Statistical
Discrimination". In: _Quarterly Journal of Economics_ 116.1, pp. 313-350. DOI:
[10.1162/003355301556329](https://doi.org/10.1162%2F003355301556329).

Angrist, J. D. and A. B. Krueger (1991). "Does Compulsory School Attendance Affect
Schooling and Earnings?" In: _Quarterly Journal of Economics_ 106.4, pp. 979-1014. DOI:
[10.2307/2937954](https://doi.org/10.2307%2F2937954).

Angrist, J. D. and J. Pischke (2009). _Mostly Harmless Econometrics: An Empiricist's
Companion_. Princeton University Press. ISBN: 0691120358.

Arcidiacono, P. (2004). "Ability Sorting and the Returns to College Major". In: _Journal
of Econometrics_ 121, pp. 343-375. DOI:
[10.1016/j.jeconom.2003.10.010](https://doi.org/10.1016%2Fj.jeconom.2003.10.010).

Arcidiacono, P., E. Aucejo, A. Maurel, et al. (2016). _College Attrition and the Dynamics
of Information Revelation_. Working Paper. Duke University. URL:
[https://tyleransom.github.io/research/CollegeDropout2016May31.pdf](https://tyleransom.github.io/research/CollegeDropout2016May31.pdf).

Arcidiacono, P., E. Aucejo, A. Maurel, et al. (2025). "College Attrition and the Dynamics
of Information Revelation". In: _Journal of Political Economy_ 133.1. DOI:
[10.1086/732526](https://doi.org/10.1086%2F732526).

Arcidiacono, P. and J. B. Jones (2003). "Finite Mixture Distributions, Sequential
Likelihood and the EM Algorithm". In: _Econometrica_ 71.3, pp. 933-946. DOI:
[10.1111/1468-0262.00431](https://doi.org/10.1111%2F1468-0262.00431).

Arcidiacono, P., J. Kinsler, and T. Ransom (2022b). "Asian American Discrimination in
Harvard Admissions". In: _European Economic Review_ 144, p. 104079. DOI:
[10.1016/j.euroecorev.2022.104079](https://doi.org/10.1016%2Fj.euroecorev.2022.104079).

Arcidiacono, P., J. Kinsler, and T. Ransom (2022a). "Legacy and Athlete Preferences at
Harvard". In: _Journal of Labor Economics_ 40.1, pp. 133-156. DOI:
[10.1086/713744](https://doi.org/10.1086%2F713744).

Arcidiacono, P. and R. A. Miller (2011). "Conditional Choice Probability Estimation of
Dynamic Discrete Choice Models With Unobserved Heterogeneity". In: _Econometrica_ 79.6,
pp. 1823-1867. DOI: [10.3982/ECTA7743](https://doi.org/10.3982%2FECTA7743).

Arroyo Marioli, F., F. Bullano, S. Kucinskas, et al. (2020). _Tracking R of COVID-19: A
New Real-Time Estimation Using the Kalman Filter_. Working Paper. medRxiv. DOI:
[10.1101/2020.04.19.20071886](https://doi.org/10.1101%2F2020.04.19.20071886).

Ashworth, J., V. J. Hotz, A. Maurel, et al. (2021). "Changes across Cohorts in Wage
Returns to Schooling and Early Work Experiences". In: _Journal of Labor Economics_ 39.4,
pp. 931-964. DOI: [10.1086/711851](https://doi.org/10.1086%2F711851).

Attanasio, O. P., C. Meghir, and A. Santiago (2011). "Education Choices in Mexico: Using a
Structural Model and a Randomized Experiment to Evaluate PROGRESA". In: _Review of
Economic Studies_ 79.1, pp. 37-66. DOI:
[10.1093/restud/rdr015](https://doi.org/10.1093%2Frestud%2Frdr015).

Aucejo, E. M. and J. James (2019). "Catching Up to Girls: Understanding the Gender
Imbalance in Educational Attainment Within Race". In: _Journal of Applied Econometrics_
34.4, pp. 502-525. DOI: [10.1002/jae.2699](https://doi.org/10.1002%2Fjae.2699).

Baragatti, M., A. Grimaud, and D. Pommeret (2013). "Likelihood-free Parallel Tempering".
In: _Statistics and Computing_ 23.4, pp. 535-549. DOI: [
10.1007/s11222-012-9328-6](https://doi.org/%2010.1007%2Fs11222-012-9328-6).

Bayer, P., R. McMillan, A. Murphy, et al. (2016). "A Dynamic Model of Demand for Houses
and Neighborhoods". In: _Econometrica_ 84.3, pp. 893-942. DOI:
[10.3982/ECTA10170](https://doi.org/10.3982%2FECTA10170).

Begg, C. B. and R. Gray (1984). "Calculation of Polychotomous Logistic Regression
Parameters Using Individualized Regressions". In: _Biometrika_ 71.1, pp. 11-18. DOI:
[10.1093/biomet/71.1.11](https://doi.org/10.1093%2Fbiomet%2F71.1.11).

Beggs, S. D., N. S. Cardell, and J. Hausman (1981). "Assessing the Potential Demand for
Electric Cars". In: _Journal of Econometrics_ 17.1, pp. 1-19. DOI:
[10.1016/0304-4076(81)90056-7](https://doi.org/10.1016%2F0304-4076%2881%2990056-7).

Berry, S., J. Levinsohn, and A. Pakes (1995). "Automobile Prices in Market Equilibrium".
In: _Econometrica_ 63.4, pp. 841-890. URL:
[http://www.jstor.org/stable/2171802](http://www.jstor.org/stable/2171802).

Blass, A. A., S. Lach, and C. F. Manski (2010). "Using Elicited Choice Probabilities to
Estimate Random Utility Models: Preferences for Electricity Reliability". In:
_International Economic Review_ 51.2, pp. 421-440. DOI:
[10.1111/j.1468-2354.2010.00586.x](https://doi.org/10.1111%2Fj.1468-2354.2010.00586.x).

Blundell, R. (2010). "Comments on: ``Structural vs. Atheoretic Approaches to
Econometrics'' by Michael Keane". In: _Journal of Econometrics_ 156.1, pp. 25-26. DOI:
[10.1016/j.jeconom.2009.09.005](https://doi.org/10.1016%2Fj.jeconom.2009.09.005).

Bresnahan, T. F., S. Stern, and M. Trajtenberg (1997). "Market Segmentation and the
Sources of Rents from Innovation: Personal Computers in the Late 1980s". In: _The RAND
Journal of Economics_ 28.0, pp. S17-S44. DOI:
[10.2307/3087454](https://doi.org/10.2307%2F3087454).

Brien, M. J., L. A. Lillard, and S. Stern (2006). "Cohabitation, Marriage, and Divorce in
a Model of Match Quality". In: _International Economic Review_ 47.2, pp. 451-494. DOI:
[10.1111/j.1468-2354.2006.00385.x](https://doi.org/10.1111%2Fj.1468-2354.2006.00385.x).

Card, D. (1995). "Using Geographic Variation in College Proximity to Estimate the Return
to Schooling". In: _Aspects of Labor Market Behaviour: Essays in Honour of John
Vanderkamp_. Ed. by L. N. Christofides, E. K. Grant and R. Swidinsky. Toronto: University
of Toronto Press.

Cardell, N. S. (1997). "Variance Components Structures for the Extreme-Value and Logistic
Distributions with Application to Models of Heterogeneity". In: _Econometric Theory_ 13.2,
pp. 185-213. URL:
[https://www.jstor.org/stable/3532724](https://www.jstor.org/stable/3532724).

Caucutt, E. M., L. Lochner, J. Mullins, et al. (2020). _Child Skill Production: Accounting
for Parental and Market-Based Time and Goods Investments_. Working Paper 27838. National
Bureau of Economic Research. DOI: [10.3386/w27838](https://doi.org/10.3386%2Fw27838).

Chen, X., H. Hong, and D. Nekipelov (2011). "Nonlinear Models of Measurement Errors". In:
_Journal of Economic Literature_ 49.4, pp. 901-937. DOI:
[10.1257/jel.49.4.901](https://doi.org/10.1257%2Fjel.49.4.901).

Chintagunta, P. K. (1992). "Estimating a Multinomial Probit Model of Brand Choice Using
the Method of Simulated Moments". In: _Marketing Science_ 11.4, pp. 386-407. DOI:
[10.1287/mksc.11.4.386](https://doi.org/10.1287%2Fmksc.11.4.386).

Cinelli, C. and C. Hazlett (2020). "Making Sense of Sensitivity: Extending Omitted
Variable Bias". In: _Journal of the Royal Statistical Society: Series B (Statistical
Methodology)_ 82.1, pp. 39-67. DOI:
[10.1111/rssb.12348](https://doi.org/10.1111%2Frssb.12348).

Coate, P. and K. Mangum (2019). _Fast Locations and Slowing Labor Mobility_. Working Paper
19-49. Federal Reserve Bank of Philadelphia.

Cunha, F., J. J. Heckman, and S. M. Schennach (2010). "Estimating the Technology of
Cognitive and Noncognitive Skill Formation". In: _Econometrica_ 78.3, pp. 883-931. DOI:
[10.3982/ECTA6551](https://doi.org/10.3982%2FECTA6551).

Cunningham, S. (2021). _Causal Inference: The Mixtape_. Yale University Press. URL:
[https://www.scunning.com/causalinference_norap.pdf](https://www.scunning.com/causalinference_norap.pdf).

Delavande, A. and C. F. Manski (2015). "Using Elicited Choice Probabilities in
Hypothetical Elections to Study Decisions to Vote". In: _Electoral Studies_ 38, pp. 28-37.
DOI: [10.1016/j.electstud.2015.01.006](https://doi.org/10.1016%2Fj.electstud.2015.01.006).

Delavande, A. and B. Zafar (2019). "University Choice: The Role of Expected Earnings,
Nonpecuniary Outcomes, and Financial Constraints". In: _Journal of Political Economy_
127.5, pp. 2343-2393. DOI: [10.1086/701808](https://doi.org/10.1086%2F701808).

Diegert, P., M. A. Masten, and A. Poirier (2025). _Assessing Omitted Variable Bias when
the Controls are Endogenous_. arXiv. DOI:
[10.48550/ARXIV.2206.02303](https://doi.org/10.48550%2FARXIV.2206.02303).

Erdem, T. and M. P. Keane (1996). "Decision-Making under Uncertainty: Capturing Dynamic
Brand Choice Processes in Turbulent Consumer Goods Markets". In: _Marketing Science_ 15.1,
pp. 1-20. DOI: [10.1287/mksc.15.1.1](https://doi.org/10.1287%2Fmksc.15.1.1).

Evans, R. W. (2018). _Simulated Method of Moments (SMM) Estimation_. QuantEcon Note.
University of Chicago. URL:
[https://notes.quantecon.org/submission/5b3db2ceb9eab00015b89f93](https://notes.quantecon.org/submission/5b3db2ceb9eab00015b89f93).

Farber, H. S. and R. Gibbons (1996). "Learning and Wage Dynamics". In: _Quarterly Journal
of Economics_ 111.4, pp. 1007-1047. DOI:
[10.2307/2946706](https://doi.org/10.2307%2F2946706).

Fu, C., N. Grau, and J. Rivera (2020). _Wandering Astray: Teenagers' Choices of Schooling
and Crime_. Working Paper. University of Wisconsin-Madison. URL:
[https://www.ssc.wisc.edu/~cfu/wander.pdf](https://www.ssc.wisc.edu/~cfu/wander.pdf).

Gillingham, K., F. Iskhakov, A. Munk-Nielsen, et al. (2022). "Equilibrium Trade in
Automobiles". In: _Journal of Political Economy_. DOI:
[10.1086/720463](https://doi.org/10.1086%2F720463).

Haile, P. (2019). _``Structural vs. Reduced Form'' Language and Models in Empirical
Economics_. Lecture Slides. Yale University. URL:
[http://www.econ.yale.edu/~pah29/intro.pdf](http://www.econ.yale.edu/~pah29/intro.pdf).

Haile, P. (2024). _Models, Measurement, and the Language of Empirical Economics_. Lecture
Slides. Yale University. URL:
[https://www.dropbox.com/s/8kwtwn30dyac18s/intro.pdf](https://www.dropbox.com/s/8kwtwn30dyac18s/intro.pdf).

Heckman, J. J., J. Stixrud, and S. Urzua (2006). "The Effects of Cognitive and
Noncognitive Abilities on Labor Market Outcomes and Social Behavior". In: _Journal of
Labor Economics_ 24.3, pp. 411-482. DOI:
[10.1086/504455](https://doi.org/10.1086%2F504455).

Hotz, V. J. and R. A. Miller (1993). "Conditional Choice Probabilities and the Estimation
of Dynamic Models". In: _The Review of Economic Studies_ 60.3, pp. 497-529. DOI:
[10.2307/2298122](https://doi.org/10.2307%2F2298122).

Hurwicz, L. (1950). "Generalization of the Concept of Identification". In: _Statistical
Inference in Dynamic Economic Models_. Hoboken, NJ: John Wiley and Sons, pp. 245-257.

Ishimaru, S. (2022). _Geographic Mobility of Youth and Spatial Gaps in Local College and
Labor Market Opportunities_. Working Paper. Hitotsubashi University.

James, J. (2011). _Ability Matching and Occupational Choice_. Working Paper 11-25. Federal
Reserve Bank of Cleveland.

James, J. (2017). "MM Algorithm for General Mixed Multinomial Logit Models". In: _Journal
of Applied Econometrics_ 32.4, pp. 841-857. DOI:
[10.1002/jae.2532](https://doi.org/10.1002%2Fjae.2532).

Jin, H. and H. Shen (2020). "Foreign Asset Accumulation Among Emerging Market Economies: A
Case for Coordination". In: _Review of Economic Dynamics_ 35.1, pp. 54-73. DOI:
[10.1016/j.red.2019.04.006](https://doi.org/10.1016%2Fj.red.2019.04.006).

Keane, M. P. (2010). "Structural vs. Atheoretic Approaches to Econometrics". In: _Journal
of Econometrics_ 156.1, pp. 3-20. DOI:
[10.1016/j.jeconom.2009.09.003](https://doi.org/10.1016%2Fj.jeconom.2009.09.003).

Keane, M. P. and K. I. Wolpin (1997). "The Career Decisions of Young Men". In: _Journal of
Political Economy_ 105.3, pp. 473-522. DOI:
[10.1086/262080](https://doi.org/10.1086%2F262080).

Koopmans, T. C. and O. Reiersol (1950). "The Identification of Structural
Characteristics". In: _The Annals of Mathematical Statistics_ 21.2, pp. 165-181. URL:
[http://www.jstor.org/stable/2236899](http://www.jstor.org/stable/2236899).

Kosar, G., T. Ransom, and W. van der Klaauw (2022). "Understanding Migration Aversion
Using Elicited Counterfactual Choice Probabilities". In: _Journal of Econometrics_ 231.1,
pp. 123-147. DOI:
[10.1016/j.jeconom.2020.07.056](https://doi.org/10.1016%2Fj.jeconom.2020.07.056).

Krauth, B. (2016). "Bounding a Linear Causal Effect Using Relative Correlation
Restrictions". In: _Journal of Econometric Methods_ 5.1, pp. 117-141. DOI:
[10.1515/jem-2013-0013](https://doi.org/10.1515%2Fjem-2013-0013).

Lang, K. and M. D. Palacios (2018). _The Determinants of Teachers' Occupational Choice_.
Working Paper 24883. National Bureau of Economic Research. DOI:
[10.3386/w24883](https://doi.org/10.3386%2Fw24883).

Lee, D. S., J. McCrary, M. J. Moreira, et al. (2020). _Valid t-ratio Inference for IV_.
Working Paper. arXiv. URL:
[https://arxiv.org/abs/2010.05058](https://arxiv.org/abs/2010.05058).

Lewbel, A. (2019). "The Identification Zoo: Meanings of Identification in Econometrics".
In: _Journal of Economic Literature_ 57.4, pp. 835-903. DOI:
[10.1257/jel.20181361](https://doi.org/10.1257%2Fjel.20181361).

Mahoney, N. (2022). "Principles for Combining Descriptive and Model-Based Analysis in
Applied Microeconomics Research". In: _Journal of Economic Perspectives_ 36.3, pp. 211-22.
DOI: [10.1257/jep.36.3.211](https://doi.org/10.1257%2Fjep.36.3.211).

McFadden, D. (1978). "Modelling the Choice of Residential Location". In: _Spatial
Interaction Theory and Planning Models_. Ed. by A. Karlqvist, L. Lundqvist, F. Snickers
and J. W. Weibull. Amsterdam: North Holland, pp. 75-96.

McFadden, D. (1989). "A Method of Simulated Moments for Estimation of Discrete Response
Models Without Numerical Integration". In: _Econometrica_ 57.5, pp. 995-1026. DOI:
[10.2307/1913621](https://doi.org/10.2307%2F1913621). URL:
[http://www.jstor.org/stable/1913621](http://www.jstor.org/stable/1913621).

Mellon, J. (2020). _Rain, Rain, Go Away: 137 Potential Exclusion-Restriction Violations
for Studies Using Weather as an Instrumental Variable_. Working Paper. University of
Manchester. URL:
[https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3715610](https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3715610).

Miller, R. A. (1984). "Job Matching and Occupational Choice". In: _Journal of Political
Economy_ 92.6, pp. 1086-1120. DOI: [10.1086/261276](https://doi.org/10.1086%2F261276).

Mincer, J. (1974). _Schooling, Experience and Earnings_. New York: Columbia University
Press for National Bureau of Economic Research.

Ost, B., W. Pan, and D. Webber (2018). "The Returns to College Persistence for Marginal
Students: Regression Discontinuity Evidence from University Dismissal Policies". In:
_Journal of Labor Economics_ 36.3, pp. 779-805. DOI:
[10.1086/696204](https://doi.org/10.1086%2F696204).

Oster, E. (2019). "Unobservable Selection and Coefficient Stability: Theory and Evidence".
In: _Journal of Business & Economic Statistics_ 37.2, pp. 187-204. DOI:
[10.1080/07350015.2016.1227711](https://doi.org/10.1080%2F07350015.2016.1227711).

Pischke, S. (2007). _Lecture Notes on Measurement Error_. Lecture Notes. London School of
Economics. URL:
[http://econ.lse.ac.uk/staff/spischke/ec524/Merr_new.pdf](http://econ.lse.ac.uk/staff/spischke/ec524/Merr_new.pdf).

Ransom, M. R. and T. Ransom (2018). "Do High School Sports Build or Reveal Character?
Bounding Causal Estimates of Sports Participation". In: _Economics of Education Review_
64, pp. 75-89. DOI:
[10.1016/j.econedurev.2018.04.002](https://doi.org/10.1016%2Fj.econedurev.2018.04.002).

Ransom, T. (2022). "Labor Market Frictions and Moving Costs of the Employed and
Unemployed". In: _Journal of Human Resources_ 57.S, pp. S137-S166. DOI:
[10.3368/jhr.monopsony.0219-10013R2](https://doi.org/10.3368%2Fjhr.monopsony.0219-10013R2).

Rudik, I. (2020). "Optimal Climate Policy When Damages Are Unknown". In: _American
Economic Journal: Economic Policy_ 12.2, pp. 340-373. DOI:
[10.1257/pol.20160541](https://doi.org/10.1257%2Fpol.20160541).

Rust, J. (1987). "Optimal Replacement of GMC Bus Engines: An Empirical Model of Harold
Zurcher". In: _Econometrica_ 55.5, pp. 999-1033. URL:
[http://www.jstor.org/stable/1911259](http://www.jstor.org/stable/1911259).

Shalizi, C. R. (2019). _Advanced Data Analysis from an Elementary Point of View_.
Cambridge University Press. URL:
[http://www.stat.cmu.edu/~cshalizi/ADAfaEPoV/ADAfaEPoV.pdf](http://www.stat.cmu.edu/~cshalizi/ADAfaEPoV/ADAfaEPoV.pdf).

Smith Jr., A. A. (2008). "Indirect Inference". In: _The New Palgrave Dictionary of
Economics_. Ed. by S. N. Durlauf and L. E. Blume. Vol. 1-8. London: Palgrave Macmillan.
DOI: [10.1007/978-1-349-58802-2](https://doi.org/10.1007%2F978-1-349-58802-2). URL:
[http://www.econ.yale.edu/smith/palgrave7.pdf](http://www.econ.yale.edu/smith/palgrave7.pdf).

Stinebrickner, R. and T. Stinebrickner (2014a). "Academic Performance and College Dropout:
Using Longitudinal Expectations Data to Estimate a Learning Model". In: _Journal of Labor
Economics_ 32.3, pp. 601-644. DOI: [10.1086/675308](https://doi.org/10.1086%2F675308).

Stinebrickner, R. and T. R. Stinebrickner (2014b). "A Major in Science? Initial Beliefs
and Final Outcomes for College Major and Dropout". In: _Review of Economic Studies_ 81.1,
pp. 426-472. DOI: [10.1093/restud/rdt025](https://doi.org/10.1093%2Frestud%2Frdt025).

Su, C. and K. L. Judd (2012). "Constrained Optimization Approaches to Estimation of
Structural Models". In: _Econometrica_ 80.5, pp. 2213-2230. DOI:
[10.3982/ECTA7925](https://doi.org/10.3982%2FECTA7925).

Train, K. (2009). _Discrete Choice Methods with Simulation_. 2nd ed. Cambridge; New York:
Cambridge University Press. ISBN: 9780521766555.

Wiswall, M. and B. Zafar (2018). "Preference for the Workplace, Investment in Human
Capital, and Gender". In: _Quarterly Journal of Economics_ 133.1, pp. 457-507. DOI:
[10.1093/qje/qjx035](https://doi.org/10.1093%2Fqje%2Fqjx035).

Young, A. (2020). _Consistency without Inference: Instrumental Variables in Practical
Application_. Working Paper. London School of Economics.
]