I consider reproducible analysis to be really important. In particular, I'd like to see more researchers in psychology:

  • sharing their data
  • producing their journal articles using literate programming technologies like Sweave and knitr that combine data analysis code and text into an integrated documenst.

I consider this particularly important in light of concerns over research fraud (e.g., data fabrication) and various forms of data fishing (e.g., testing many hypotheses and only reporting the significant one).

Certainly in statistical fields, there is a lot of discussion about the importance of reproducible analyses (e.g., see here, and here). However, I sometimes feel that in psychology this is only starting to filter through.

Thus, my question is:

  • Are there any journal articles in psychology that have advocated reproducible analysis?

There are many angles that such articles might take: reward structures that discourage open science and reproducible analysis; technical challenges which make it difficult to do such analyses; how-to articles that outline how to perform reproducible analyses in psychology. I'd be interested in any articles that address these topics.

  • $\begingroup$ I don't own a copy of the APA manual myself, so I couldn't look this up, but this is a quote from a paper by Bakker & Wicherts (2011), The (mis)reporting of statistical results in psychology journals: "...since standard 8.14 of the ethical standards of the American Psychological Association (2010) states that data should be shared after research results are published...". I thought this might interested you. $\endgroup$
    – Mien
    Jun 18, 2012 at 21:31
  • $\begingroup$ This reanalysis, published in the Reproducibility and Robustness gateway on F1000Research, undertakes to reproduce (part of) an earlier published report, in order to criticize its methods. (A fuller story is available in blog form here.) A special point of interest is that the reproduction had to be done in SAS! All of the reanalysis code is publicly available. $\endgroup$ Sep 7, 2017 at 23:47

1 Answer 1


It's not a journal article, but it's a project, called Reproducibility Project. They are re-doing psychological studies to estimate the reproducibility of a sample of studies. They try to stay as close as possible on the original study (they e-mail the author(s) for the exact stimuli etc.). Here's the main site and here's the link to their spreadsheet.

This is a short article promoting the sharing of data in psychology. Another related study (by the same author) is also mentioned.

  • $\begingroup$ It is PLoS (Public Library of Science) and it is not a journal, but it publishes several journals that have names that start with PLoS. None of these journals are psychology specific. Further, these journals don't "claim to be 'open-access'". They basically defined what it means to be open-access. However, I have not heard of them trying to promote reproducible data analysis, especially in psychology (they concentrate on bio and medicine mostly, although PLoS One does publish some other things); I don't see how your last paragraph has relevance. $\endgroup$ May 17, 2012 at 21:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.