Quasi-experiments again

Instead of using carefully adjusted DAGs,
we can use context to isolate/identify the pathway between
treatment and outcome in observational data

Diff-in-diff was one kind of quasi-experiment

Treatment/control + before/after

Regression discontinuity designs (RDD) are another

Arbitrary rules determine access to programs

Rules to access programs

Lots of policies and programs are
based on arbitrary rules and thresholds

If you're above the threshold, you're in the program;
if you're below, you're not (or vice versa)

Key terms

Running / forcing variable

Index or measure that determines eligibility

Cutoff / cutpoint / threshold

Number that formally assigns access to program

Discontinuities everywhere!

Hypothetical tutoring program

Students take an entrance exam

Those who score 70 or lower
get a free tutor for the year

Students then take an exit exam
at the end of the year

Causal inference intuition

The people right before and right after the threshold are essentially the same

Causal inference intuition

The people right before and right after the threshold are essentially the same

Pseudo treatment and control groups!

Compare outcomes for those
right before/after, calculate difference

Geographic discontinuities

Geographic discontinuities

Time discontinuities

Time discontinuities

Delivering at 12:01 AM makes you stay longer in the hospital…

Time discontinuities

Test score discontinuities

Test score discontinuities

Cutoff seems rule-based

Earnings are slightly higher

RDDs are all the rage

People love these things!

They're intuitive, compelling, and highly graphical

Main goal of RD

Measure the gap in outcome for
people on both sides of the cutpoint

Gap = δ =
local average treatment effect (LATE)

Drawing lines

The size of the gap depends on how
you draw the lines on each side of the cutoff

The type of lines you choose can
change the estimate of δ—sometimes by a lot!

There's no one right way to draw lines!

Line-drawing considerations

Parametric vs. non-parametric lines

Measuring the gap

Bandwidths

Kernels

Parametric lines

Formulas with parameters

Parametric lines

Not just for straight lines!
Make curvy with exponents or trigonometry

Parametric lines

It's important to get the parameters right!

Line should fit the data pretty well

Nonparametric lines

Lines without parameters

Use the data to find the best line,
often with windows and moving averages

Locally estimated/weighted scatterplot smoothing (LOESS/LOWESS)
is a common method (but not the only one!)

Measuring gap with parametric lines

Measuring gap with parametric lines

Easiest way: center the running variable around the threshold

Measuring gap with parametric lines

program_data <- tutoring %>% 
  mutate(entrance_centered = 
           entrance_exam - 70)
model1 <- lm(exit_exam ~ 
               entrance_centered + tutoring,
             data = program_data)

tidy(model1)

## # A tibble: 3 × 3
##   term              estimate std.error
##   <chr>                <dbl>     <dbl>
## 1 (Intercept)         59.3      0.440 
## 2 entrance_centered    0.514    0.0268
## 3 tutoringTRUE        11.0      0.802

Measuring gap with nonparametric lines

Can't use regression; use rdrobust R package

Measuring gap with nonparametric lines

rdrobust(y = tutoring$exit_exam, x = tutoring$entrance_exam, c = 70)

## =============================================================================
##         Method     Coef. Std. Err.         z     P>|z|      [ 95% C.I. ]       
## =============================================================================
##   Conventional    -9.992     1.708    -5.852     0.000   [-13.339 , -6.646]    
##         Robust         -         -    -4.992     0.000   [-14.244 , -6.212]    
## =============================================================================

Bandwidths

All you really care about is the
area right around the cutoff

Observations far away don't matter
because they're not comparable

Bandwidth = window around cutoff

Bandwidths

Algorithms exist to choose optimal width

Also use common sense

Maybe ±5 for the entrance exam?

For robustness, check what happens
if you double and halve the bandwidth

Kernels

Because we care the most about
observations right by the cutoff,
give more distant ones less weight

Kernel = method for assigning importance to
observations based on distance to the cutoff

Try everything!

Your estimate of δ depends on all these:

Line type (parametric vs. nonparametric)

Try lots of different combinations!

It's greedy!

You need lots of data,
since you're throwing most of it away

It's limited in scope!

You're only measuring the ATE
for people in the bandwidth

Local Average Treatment Effect (LATE)

It's limited in scope!

You can't make population-level
claims with a LATE

(But can you really do that with RCTs or diff-in-diff?)

"The realistic conclusion to draw is that
all quantitative empirical results
that we encounter are 'local'"

Angrist and Pischke, Mostly Harmless Econometrics, pp. 23–24

Graphics are neat!

Which gaps are significant?

All of them!

Don't rely only on graphics

Manipulation!

People might know about the cutoff
and change their behavior

People might fudge numbers or work to
cross the threshold to get in/out of program

If so, those right next to the cutoff are
no longer comparable treatment/control groups

Manipulation!

Check with a McCrary density test

rddensity::rdplotdensity() in R

Noncompliance!

People on the margin of the cutoff
might end up in/out of the program

The ACA, subsidies, Medicaid, and 138% of the poverty line

Sharp vs. fuzzy discontinuities

Sharp discontinuity

Perfect compliance

Fuzzy discontinuity

Imperfect compliance

Fuzzy discontinuities

Address noncompliance with
instrumental variables
(more on this later!)

Use an instrument for which side
of the cutoff people should be on

Effect is only for compliers near the cutoff
(complier LATE; doubly local effect)