Abstract
Meta‐analysis (MA), a powerful tool for synthesizing reported results, is influential in ecology. While ecologists have long been well‐informed on the potential problems associated with nonindependence in experimental work (e.g., pseudoreplication), they have, until recently, largely neglected this issue in MA. However, results used in MAs are likely much more similar when they come from the same locality, system, or laboratory. A simple and common form of nonindependence in MA arises when multiple data points, that is, observed effect sizes, come from the same paper. We obtained original data from 20 published MAs, reconstructed the published analyses, and then, for 14 that had not accounted for a paper effect, used three approaches to evaluate whether within‐paper nonindependence was a problem. First, we found that “nonsense” explanatory variables added to the original analyses were statistically significant ( p n = 14 MAs) but added a random paper effect to each revised analysis. In 12 out of the 14 MAs, an added random effect was statistically significant (indicating group nonindependence that was not accounted for in the original analyses), and often the original inferences were substantially altered. Further, incorporating random paper effects was not a sufficient solution to nonindependence. Thus, problems resulting from nonindependence are often substantial, and accounting for the problem will likely require careful consideration of the details of the potential dependence among observed effect sizes. MAs that do not properly account for this problem may reach unwarranted conclusions.