Propensity score matching

In statistics, propensity score matching (PSM) is one of quasi-empirical “correction strategies” that corrects for the selection biases in making estimates.

Overview

PSM is for cases of causal inference and simple selection bias in non-experimental settings in which: (i) few units in the non-experimental comparison group are comparable to the treatment units; and (ii) selecting a subset of comparison units similar to the treatment unit is difficult because units must be compared across a high-dimensional set of pretreatment characteristics.

In normal Matching we match on single characteristics that distinguish treatment and control groups (to try to make them more alike). But If the two groups do not have substantial overlap, then substantial error may be introduced: E.g., if only the worst cases from the untreated “comparison”group are compared to only the best cases from the treatment group, the result may be regression toward the mean which may makes the comparison group look betteror worse than reality.

PSM employs a predicted probability of group membership e.g., treatment vs. control group--based on observed predictors, usually obtained from logistic regression to create a counterfactual group. Also propensity scores may be used for matching or as covariates—alone or with other matching variables or covariates.

History

In 1983, Rosenbaum and Rubin published their seminal paper that first proposed this approach. From the 1970s, Heckman and his colleagues focused on the problem of selection biases, and traditional approaches to program evaluation, including randomized experiments, classical matching, and statistical controls. Heckman later developed difference in differences method.

General procedure

1.Run logistic regression:
*Dependent variable: "Y" = 1, if participate; "Y" = 0, otherwise.
*Choose appropriate conditioning (instrumental) variables.
*Obtain propensity score: predicted probability ("p") or log ["p"/(1 − "p")] .

2.Match each participant to one or more nonparticipants on propensity score:
*Nearest neighbor matching
*Caliper matching
*Mahalanobis metric matching in conjunction with PSM
*Stratification matching
*Difference-in-differences matching (kernel and local linear weights)

3.Multivariate analysis based on new sample

Requirements for a good PSM

*Identify treatment and comparison groups with substantial overlap
*Match, as much as possible, on variables that are precisely measured and stable (to avoid extreme baseline scores that will regress toward the mean)
*Use a composite variable—e.g., a propensity score—which minimizes group differences across many scores

For example:

Disadvantages

Howard Bloom, MDRC, sees PSM as a somewhat improved version of simple matching, but with many of the same limitations
*Inclusion of propensity scores can help reduce large biases, but significant biases may remain
*Local comparison groups are best—PSM is no miracle maker (it cannot match unmeasured contextual variables)
*Short-term biases (2 years) are substantially less than medium term (3 to 5 year) biases—the value of comparison groups may deteriorate

Michael Sosin, University of Chicago also identifies following problems with PSM:
*Strong assumption that untreated cases were not treated at random
*Argues for using multiple methods and not relying on PSM

Shadish, Cook, & Campbell (2002) identifies further shortcomings with PSM:
*Large samples are required
*Group overlap must be substantial
*Hidden bias may remain because matching only controls for observed variables (to the extent that they are perfectly measured)

In general risks of PSM include:
*They may undermine the argument for experimental designs—an argument that is hard enough to make,
*They may be used to act “as if” a panel survey is an experimental design, overestimating the certainty of findings based on the PSM.

References

* [http://ssw.unc.edu/VRC/Lectures/PSM_SSWR_2004.pdf]
* [http://fmwww.bc.edu/RePEc/usug2001/psmatch.pdf]