Are meta-analyses truly as good as the original research?

The reason why Glass (1976) distinguished between primary, secondary and meta-analysis is that often the original data is unavailable. This has to do with many issues such as anonymity and confidentiality. The methodology approach of meta-analysis overcomes this problem by using each study’s methodology and findings to investigate and compare the effect(s) of an influence/intervention.

There is a whole compendium of risks to interpretation to statistics and these are not unique to meta-analyses.  For example, Black and Wiliam (1998) noted that an effect size can be influenced by the range of achievement in the population. “An increase of 5 points on a test where the population standard deviation is 10 points would result in an effect size of 0.5 standard deviations. However, the same intervention when administered only to the upper half of the same population, provided that it was equally effective for all students, would result in an effect size of over 0.8 standard deviations, due to the reduced variance of the subsample. An often-observed finding in the literature—that formative assessment interventions are more successful for students with special educational needs (for example in Fuchs & Fuchs, 1986)—is difficult to interpret without some attempt to control for the restriction of range, and may simply be a statistical artefact. But this problem with restriction of range can occur in primary, secondary and meta-analyses.  Campbell and Stanley (1963) highlight many other possible threats to the validity of interpretation of statistics, no matter whether primary, secondary or meta-analysis is used.