Manipulation checks

Manipulation checks

Manipulation check is a term in experimental research in the social sciences which refers to certain kinds of secondary evaluations of an experiment.

Overview

In experiments, an experimenter manipulates some aspect of a process or task and randomly assigns subjects to different levels of the manipulation ("experimental conditions"). The experimenter then observes whether variation in the manipulated variables cause differences in the dependent variable.

Manipulation checks are separate measured variables that show what the manipulated variables concurrently affect (besides the dependent variable of interest). Manipulations are NOT intended to verify that the manipulated factor caused variation in the dependent variable. Random assignment, manipulation before measurement of the dependent variable, and statistical tests of effect of the manipulated variable on the dependent variable verify this. Thus, a failed manipulation check does not refute that the manipulation caused variation in the dependent variable.

In contrast, a successful manipulation check can help an experimenter rule out reasons why a manipulation may have failed to influence an independent variable. When a manipulation creates significant differences between experimental conditions in both (1) the dependent variable and (2) the measured manipulation check variable, the interpretation is that (1) the manipulation "causes" variation in the dependent variable (the "effect") and (2) the manipulation also explains variation in some other, more theoretically obvious measured variable that it is expected to concurrently influence, which assists in interpreting the "cause" (i.e., it only help interpret the "cause"; it is not necessary to affirm that the "cause" causes an effect).

See also