Who's at the Cutoff? Characterizing the Analytical Sample of Regression Discontinuity Designs
P8-S211-2
Presented by: Bryant Moy
Social scientists have long recognized that unrepresentative samples limit our ability to draw conclusions about target populations. Nevertheless, researchers routinely employ Regression Discontinuity Designs (RDD) in applied causal inference work without characterizing who comprises their analytical sample. Although researchers explicitly acknowledge that RDD results are only identified near the cutoff, they rarely provide a discussion analyzing whom this effect is estimated upon. This oversight has significant implications: we cannot assess whether findings generalize beyond the cutoff, we cannot improve theory development, and we may provide misguided policy recommendations. We propose a novel framework for characterizing RDD analytical samples and comparing them against the broader study population or other target populations of interest. Our approach involves computing covariate means weighted by distance to the cutoff and providing researchers with graphical tools to understand who contributes to the estimated effects. We demonstrate our method's utility through empirical applications and provide an R package for implementation. Our project highlights the importance of description in causal inference work.
Keywords: Regression Discontinuity Design, Local Average Treatment Effect, Description, Causal Inference