I'm looking for references related to how researchers and laypeople understand concepts of statistical inference. In particular, I'm looking for articles in which subjects could demonstrate statistical competence by evaluating statistical claims. Does anyone have a relevant reference?
$\begingroup$
$\endgroup$
6
-
2$\begingroup$ Even among researchers there is widespread misunderstanding of core statistics ideas. Look at the work by Geoff Cumming. Example paper title: 'Researchers misunderstand confidence intervals and standard error bars.' $\endgroup$– TomApr 14, 2015 at 13:19
-
$\begingroup$ Thanks, this is exactly what I'm looking for. I would certainly upvote your comment as an answer. $\endgroup$– JoshApr 14, 2015 at 13:20
-
$\begingroup$ This wonderful TED talk may be of interest. $\endgroup$– lemonApr 15, 2015 at 10:23
-
1$\begingroup$ The general consensus in statistical psychology is that lay people and researchers understand statistics in the same way, generally speaking, and that way is "poorly." (Joke disclaimer.) $\endgroup$– Christian HummeluhrApr 16, 2015 at 9:58
-
1$\begingroup$ @tom good point. another one i love is the statement "approached statistical significance" ... moreover, i recall reading a review of journals that showed very few studies having reported testing for normality in parametric analyses. i also wish more studies would report effect size. it's quite funny how research publication obliges one to stick to the statistical reporting conventions of the field/journal, no matter how inappropriate such conventions may be. $\endgroup$– faustusMay 10, 2015 at 11:20
|
Show 1 more comment
2 Answers
$\begingroup$
$\endgroup$
Even among researchers there is widespread misunderstanding of core statistics ideas. Look at the work by Geoff Cumming. Example paper title: 'Researchers misunderstand confidence intervals and standard error bars.'
$\begingroup$
$\endgroup$
2
To add to Tom's answer and expand on my comment that lay people and researchers both generally have an inadequate understanding of basic statistics, another study by Hoekstra, Morey, Rouder and Wagenmakers (2014) asked 120 researchers and grad students plus 442 first-year students in psychology to indicate whether the following six different interpretations of a 95% confidence interval were correct.
Questionnaire content (Hoekstra, Morey, Rouder and Wagenmakers, 2014)
- The probability that the true mean is greater than 0 is at least 95 %.
- The probability that the true mean equals 0 is smaller than 5 %.
- The “null hypothesis” that the true mean equals 0 is likely to be incorrect.
- There is a 95 % probability that the true mean lies between 0.1 and 0.4.
- We can be 95 % confident that the true mean lies between 0.1 and 0.4.
- If we were to repeat the experiment over and over, then 95 % of the time the true mean falls between 0.1 and 0.4.
(I highly recommend looking up the quite humorous questionnaire, featuring Professor Bumbledorf, in Appendix 2.
On average, participants endorsed more than three of these statements, and all three subgroups gave substantial endorsement for individual statements. All of the statements are wrong, however. Only eight first-year students and three researchers gave a flawless response, i.e., all false, despite the questionnaire explicitly raising this possibility!
The correct interpretation is, “If we were to repeat the experiment over and over, then the confidence intervals contain the true mean 95% of the time.”
(Mouse-over for the correct interpretation.)
References
answered May 10, 2015 at 9:21
-
$\begingroup$ Is there anything about how statistics in psychology are taught differently in various places? I noticed this was at the univ. of Amsterdam. I did my BSc in the UK and we aren't really taught statistics, more how to use SPSS $\endgroup$ May 10, 2015 at 10:52
-
$\begingroup$ @queenslug Anecdotally, I would say that sums up most undergrad statistics. (But don't get me started.) Also, given that many of the researchers tested were not trained at the UvA, only working there, I would say there's every reason to suspect that it's fairly representative. $\endgroup$ May 10, 2015 at 11:01