Lecture 6 Evaluation Designs: Quasi Experiments
Gang He
October 6, 2026
Recap lecture 5
- Evaluation design
- RCT
- Case: Microfinance and RCT
- Proposal
Today’s agenda
- Quasi-experimental design
- Regression discontinuity
- Difference in differences
- Matching
- Case:
- Air quality and life expectancy
- Alcohol consumption and health
Capturing the wedge
Source: Andrew Wheiss
Evaluation designs
- Experimental
- Randomized controlled trial
- Quasi-experimental
- Instrumental variable (IV)
- Regression discontinuity (RDD)
- Difference in differences (DiD)
- Observational (Non-experimental)
- Time–series analysis (Pre- and post-intervention studies)
- Cross-sectional surveys
- Case studies
Instrumental variable design
An instrumental variable:
- It influences the likelihood of participating in a program
- Outside of the participant’s control
- Unrelated to the participant’s characteristics
Examples:
- Test scores
- Geographic proximity
- Month of birth
- Rainfall
RDD design
Application: programs that have a continuous eligibility index with clearly defined eligibility threshold to determine who is eligible and who is not.
- The index must rank people or units in continuous way.
- The index must have a clearly defined cutoff score.
- The cutoff must be unique to the program of interest.
- The score of particular individual or unit cannot be manipulated.
RDD example: Farm baseline
Source: Gertler et al. (2016)
RDD example: Farm followup
Source: Gertler et al. (2016)
Heating, air quality, and life-expectancy
Source: Chen et al. (2013)
DiD design
- Compares the CHANGES in outcomes over time between the treatment group the comparison group.
- First Difference: time-varying factors
- Second Difference: treatment effects
- Trend differences
DiD example: Employment
Source: Gertler et al. (2016)
Equal trends in DiD
Source: Gertler et al. (2016)
Testing for equal trends in DiD
- Repeated comparisons: year -2, -1, and 0
- Placebo test: a fake treatment group
- Fake treatment group + Fake outcome
- Different comparison group
Matching
Matching uses large data sets and statistical techniques to construct the best possible comparison group based on observed characteristics.
For every possible unit under treatment, it attempts to fi nd a nontreatment unit (or set of nontreatment units) that has the most similar characteristics possible.
Curse of dimensionality
Increased matching dimensions reduces matching possibilities.
Source: Gertler et al. (2016)
Propensity score matching
Uses a probability score to find the best possible comparison group.
Source: Gertler et al. (2016)
Key issues in matching
- Matching can use only observed characteristics
- Using only characteristics that are not affected by the program
- Estimates are only as good as the characteristics used for matching (understanding the criteria for participation)
Comparing evaluation designs
| Method | Source of Counterfactual | Key Assumption |
|---|---|---|
| Randomized Experiment (RCT) | Randomly assigned control group | Random assignment creates comparable treatment and control groups |
| Instrumental Variables (IV) | External variation in treatment generated by an instrument | Instrument affects treatment and affects the outcome only through treatment |
| Regression Discontinuity (RD) | Units immediately on opposite sides of a cutoff | Outcomes would change smoothly at the cutoff in the absence of treatment |
| Difference-in-Differences (DiD) | Change over time in a comparison group | Treatment and comparison groups would have followed parallel trends without treatment |
| Matching | Observationally similar untreated units | No important unobserved confounders after conditioning on observed characteristics |
References
Chen, Yuyu, Avraham Ebenstein, Michael Greenstone, and Hongbin Li. 2013. “Evidence on the Impact of Sustained Exposure to Air Pollution on Life Expectancy from China’s Huai River Policy.” Proceedings of the National Academy of Sciences, ahead of print, July. https://doi.org/10.1073/pnas.1300018110.
Gertler, Paul J., Sebastian Martinez, Patrick Premand, Laura B. Rawlings, and Christel M. J. Vermeersch. 2016. Impact Evaluation in Practice, Second Edition. World Bank. https://doi.org/10.1596/978-1-4648-0779-4.