Meta-analytic thinking

Thompson (2002, p.28) defines meta-analytic thinking as, "a) the prospective formulation of study expectations and design by explicitly invoking prior effect size measures and b) the retrospective interpretation of new results, once they are in hand, via explicit, direct comparison with the prior effect sizes in the related literature."

Prospective formulation involves reviewing the literature for other relevant studies that provide insight into the estimated effect size. Meta-analyses can be particularly useful for this purpose.

Retrospective interpretation deprecates the results of single studies in favour of patterns across multiple studies. The approach is somewhat Bayesian in its orientation.Fact|date=February 2008 That is, the results of the focal study are interpreted within a context of prior research. If the prior research is stronger and more relevant than the focal study, then the focal study is given less weight in the long-run assessment of all such studies.

The overall meta-analytic emphasis is to align research questions with estimates of population parameters. These parameters might be correlations (e.g., correlation between intelligence and job performance), standardised group differences (e.g., the effectiveness of a drug in reducing depression relative to a control group). Any one study will get a point estimate of the effect size, and this the most commonly reported result (e.g., the correlation between x and y is .5). Meta analytic thinking emphasises the importance of also obtaining confidence intervals around effect sizes (e.g., we can be 95% confident that the correlation between x and y is between .3 and .65). This approach highlights the uncertainty associated with our knowledge of the practical importance of an effect. As studies accumulate and sample sizes increase, confidence intervals get progressively smaller.

References

* Thompson, B. (2002). What future quantitative social science research could look like: Confidence intervals for effect size. Educational Researcher, 31(3), 25.

ee also

* Meta-analysis
* Effect size