3
$\begingroup$

I'm looking for examples of studies where the result is highly significant but the effect is so tiny that it is meaningless in a practical sense. These will be shared with undergraduate psychology students to illustrate that statistically significant doesn't always mean important or consequential.

$\endgroup$
  • 3
    $\begingroup$ There's the recent Facebook emotion contagion study (e.g., "P < 0.001, Cohen’s d = 0.02" and "P = 0.007, d = 0.001"). $\endgroup$ – mrt Nov 11 '15 at 19:00
  • $\begingroup$ Problem might get worse if you include CIs - as many CIs reported in studies with low p values include very low effect sizes. $\endgroup$ – jona Nov 12 '15 at 9:05
3
$\begingroup$

One example might be the effect of birth order on intelligence. Given a large enough sample there highly significant effects (p<.00001). However, these effects are very very small.

For a recent example, see Rohrer, Egloff, and Schmukle (2015): the effect of one birth order position was only one tenth of a standard deviation (1.5 IQ points)

To illustrate this small effect, in our between-family sample of sibships of two, a randomly picked firstborn had a 52% chance of having a higher IQ than a randomly picked secondborn; conversely, a secondborn had a 48% chance of having a higher IQ than a firstborn.

References

Rohrer, J. M., Egloff, B., & Schmukle, S. C. (2015). Examining the effects of birth order on personality. Proceedings of the National Academy of Sciences, 201506451. http://doi.org/10.1073/pnas.1506451112

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.