0
$\begingroup$

I'm looking for resources of reliable research in psychology. I mean 'reliable' in the following two senses:

1. Replicability

It seems more and more of the famous effects that I, as a psychology-curious layman, learned about by watching intro psych lectures or in popular media fail replication. It starts to seem like anything one believes to know about psychology is as likely as not to be false. As a layman it is quite difficult to track down all of the possible effects and replications thereof. Is there some resource which tracks the replication status of psychological research and collects the different effects which are reproducible (and which aren't)?

EDIT: I was able to find the following which is actually pretty close to what I was looking for. This is not a complete list but it is the kind of thing that I am after.

https://osf.io/fgjvw/

I found this through the links in the accepted answer to this SE question, which was suggested by a commenter below. However, now the comment is gone for some reason so I cannot give credit.

2. "Bi-partisanship"

Some areas of psychology are politically charged. Things like gender, race, intelligence, etc., are understandably controversial. It seems that often professional psychologists fall down on either side of a given issue, each insisting that it was settled by the scientific community long ago in their favour. As a layman, this is rather frustrating to make sense of. Is there some "common ground" resource which tracks the research that all sides agree on?

$\endgroup$
  • $\begingroup$ With regard to replicability are you talking about wanting a resource which tracks the scientific standpoint of psychology? $\endgroup$ – Chris Rogers Jun 16 '18 at 6:46
  • $\begingroup$ I'm not entirely sure what scientific standpoint specifically entails, but what I have in mind is a database that would collect the different effects proposed in psychology and track well how they replicate. At the basic level it could be just a number of successful/failed replications, but ideally also with discussion/nuance. Basically a resource that would let you assess the confidence we have in these various effects. $\endgroup$ – Philo Jun 16 '18 at 12:34
  • $\begingroup$ The database linked at osf.io is an up to date version of the database linked in the accepted answer you linked and is part of Open Science Collaboration (2015) article mentioned by @steveLangsford. The original and updated version is at osf.io/ezcuj/wiki/home $\endgroup$ – Chris Rogers Jun 19 '18 at 13:09
2
$\begingroup$

'psychology' is too big for this in general. But lots of people do care a lot about replication and there have been some nice many-labs efforts to asses the state of replicability across the field. Klein, et al. (2014). was one attempt.

This study is taking the usual logic of inference about a population (in this case 'effects') from a small sample, so it has the usual problem of leaving uncertain how representative the sample is of the population. For what it's worth, I think their results suggest that the size of possible 'partisanship' bias mentioned in this question is small relative to 'some effects are small and unreliable but people chase them anyway'. Of course, science media may be choosing which of these leprechaun-chasing exercises they want to report on in a partisan way, but that's a different question.

As far as I know, there is NO common ground analogous to the standard model in physics. Although many smart people have tried, as of 2018 it's still the wild west out there. The linked project certainly isn't a common-ground reference, its scope is modest (ie realistic) and it's just one of a small family of similar studies, eg see also Open Science Collaboration (2015) or Ioannidis, et al. (2014).

References

Ioannidis, J. P., Munafo, M. R., Fusar-Poli, P., Nosek, B. A., & David, S. P. (2014). Publication and other reporting biases in cognitive sciences: detection, prevalence, and prevention. Trends in cognitive sciences, 18(5), 235-241.
DOI: 10.1016/j.tics.2014.02.010
PMCID: PMC4078993
PDF: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4078993/pdf/nihms579109.pdf

Klein, R. A., Ratliff, K. A., Vianello, M., Adams Jr, R. B., Bahník, Š., Bernstein, M. J., ... & Cemalcilar, Z. (2014). Investigating variation in replicability: A “many labs” replication project. Social psychology, 45(3), 142.
DOI: 10.1027/1864-9335/a000178
PDF: http://psycnet.apa.org/fulltext/2014-20922-002.pdf
Updated 2017: https://osf.io/wx7ck/

Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. Science, 349(6251), aac4716.
DOI: 10.1126/science.aac4716
PDF: https://osf.io/447b3/download?format=pdf

$\endgroup$
  • $\begingroup$ Thanks. The Open Science Framework is quite helpful. Table 2 in osf.io/wx7ck is precisely the sort of thing I am after but larger in scope. Basically a systematic collection of all of effects where replication have been attempted so far. I've found a similar collection with a larger sample of studies (though still incomplete) from the Repoducibility Project which I have linked in my original question. As you were the closest to what I was looking for I will accept your answer. $\endgroup$ – Philo Jun 19 '18 at 13:03
3
$\begingroup$

Short answer

There is no official database I can find that collects the different effects proposed in psychology and track how they replicate. However, reputable psychology papers are published in professional journals and are re-examined all the time with meta-analyses and further research, also published in professional journals.

Professional psychologists, psychiatrists and therapists are required to maintain an up-to-date knowledge of psychological research as part of their professional codes of conduct and ethics.

Long Answer

Replicability

With regard to replicability within your question, if I am not mistaken, you are referring to finding the demarcation between science and pseudoscience within psychology as referred to by Karl Popper, in his book The Logic of Scientific Discovery (1959)

I would like to first of all point out that although there is a lot of argument which states that a large proportion of psychology is non-scientific, I would direct you to Sven Ove Hansson (2013) who argues the fact that in order for any research in psychology to be non-scientific (pseudoscientific) it has to be non-scientific when a PsychologyToday article (2016) points out that

psychology was defined by the application of scientific method(s) and psychologists conduct valuable research and have developed some key insights into animal behavior, cognition, consciousness, and the human condition.

I would also point you to my extensive answer on the subject in meta which you can read alongside other's viewpoints.

"Bi-partisanship"

As for "Bi-partisanship", I would argue that professional psychologists, psychiatrists and therapists do not fall down on either side of a given issue as intimated in your question. Whilst there are a select few, those who do are unprofessional as they are prejudicial and therefore do not abide by professional codes of conduct and ethics, which include:

  • General Principle E: Respect for People's Rights and Dignity
    of the American Psychological Association (APA) Code of Conduct (APA, 2017)

    ...Psychologists are aware of and respect cultural, individual, and role differences, including those based on age, gender, gender identity, race, ethnicity, culture, national origin, religion, sexual orientation, disability, language, and socioeconomic status, and consider these factors when working with members of such groups. Psychologists try to eliminate the effect on their work of biases based on those factors, and they do not knowingly participate in or condone activities of others based upon such prejudices.

  • Clause 22a-h of the Good Practice section
    of the British Association for Counselling and Psychotherapy (BACP) Ethical Framework for the Counselling Professions (BACP, 2018)
    1. We will respect our clients as people by providing services that:
      a. endeavour to demonstrate equality, value diversity and ensure inclusion for all clients
      b. avoid unfairly discriminating against clients or colleagues
      c. accept we are all vulnerable to prejudice and recognise the importance of self-inquiry, personal feedback and professional development
      d. work with issues of identity in open-minded ways that respect the client’s autonomy and be sensitive to whether this is viewed as individual or relational autonomy
      e. challenge assumptions that any sexual orientation or gender identity is inherently preferable to any other and will not attempt to bring about a change of sexual orientation or gender identity or seek to suppress an individual’s expression of sexual orientation or gender identity
      f. make adjustments to overcome barriers to accessibility, so far as is reasonably possible, for clients of any ability wishing to engage with a service
      g. recognise when our knowledge of key aspects of our client’s background, identity or lifestyle is inadequate and take steps to inform ourselves from other sources where available and appropriate, rather than expecting the client to teach us
      h. are open-minded with clients who appear similar to ourselves or possess familiar characteristics so that we do not suppress or neglect what is distinctive in their lives. challenge assumptions that any sexual orientation or gender identity is inherently preferable to any other and will not attempt to bring about a change of sexual orientation or gender identity or seek to suppress an individual’s expression of sexual orientation or gender identity
  • Delivering a Service section
    of National Counselling Society (NCS) Code of Ethical Practice (NCS, 2018).

    All Practitioners undertake to:

    1. Work in ways that promote client autonomy and well-being and that maintain respect and dignity for the client.
    2. Demonstrate a fully developed, professional awareness of diversity issues; and specifically not permit considerations of religion, nationality, gender, sexual orientation, marital status, age, disability, politics or social standing to adversely influence client treatment. (See Appendix B)

    Appendix B: Conversion (Reparative) Therapy

    We take the view that the practice of conversion therapy has no place in the modern world. It is unethical and harmful and not supported by evidence.

    Conversion Therapy is the term for therapy that assumes certain sexual orientations or gender identities are inferior to others, and seeks to change or suppress them on that basis.

    Sexual orientations and gender identities are not mental health disorders, although exclusion, stigma and prejudice may precipitate mental health issues for any person subjected to these abuses. Anyone accessing therapeutic help should be able to do so without fear of judgement or the threat of being pressured to change a fundamental aspect of who they are.

    1. The Society respects sexual diversity as part of our approach to diversity, equalities and social responsibility.
    2. The Society does not consider homosexuality, bisexuality, transsexual and transgendered states, or asexuality to be pathologies, mental disorders or indicative of developmental arrest. These are not ‘symptoms’ to be treated by counsellors in the sense of attempting to change or remove them. Counsellors must at all times respect the best interests of their clients.
    3. Practitioners must recognise the limits of their practice, training and experience in issues of sexuality and if necessary refer the client to an experienced therapist for specific help.
    4. If a practitioner’s personal, theoretical or religious beliefs mean that they are unable to work in a non-judgmental way with a lesbian, gay, bisexual or transgender client, or one who identifies as asexual, the client should be referred to another therapist.
    5. No member should offer counselling that seeks to change sexual orientation or influence gender identity (SOCE, reparative, conversion or reorientation therapy, or similar therapies by other names).
    6. Counselling should not be offered that seeks to eliminate or reduce same sex attraction in clients.

    Please contact the Society for further information if necessary.

Clause 22a-h of the Good Practice section of the BACP Ethical Framework for the Counselling Professions, and Appendix B of the NCS Code of Ethical Practice follows the UK Council for Psychotherapy (UKCP) Memorandum of Understanding (MoU) on Conversion Therapy in the UK, signed by many professional registering bodies, including the NCS, BACP, NHS, The British Psychological Society, and the Royal College of General Practitioners (UKCP, 2017).

To summarise

Whilst there is no official database that would collect the different effects proposed in psychology and track how they replicate, reputable psychology papers are published in professional journals and are re-examined all the time with meta-analyses and further research, also published in professional journals.

References

APA (2017). Ethical Principles of Psychologists and Code of Conduct [Online]
Available at: http://www.apa.org/ethics/code
Free PDF: http://www.apa.org/ethics/code/ethics-code-2017.pdf

BACP (2018). Ethical Framework for the Counselling Professions [Online]
Available at: https://www.bacp.co.uk/events-and-resources/ethics-and-standards/introducing-the-new-ethical-framework-2018
Free PDF: https://www.bacp.co.uk/media/3103/bacp-ethical-framework-for-the-counselling-professions-2018.pdf

Hansson, S. O. (2013). Defining pseudoscience and science. In: Pigliucci, M. & Boudry, M. (Eds.) The philosophy of pseudoscience, Chicago IL: University of Chicago Press. pp. 61-77.
Fully Readable in: Google Books at https://books.google.co.uk/books?id=Pc4OAAAAQBAJ&lpg=PA61&dq=Hansson%2C%20S.%20O.%20(1996).%20Defining%20Pseudoscience.&lr&pg=PA61#v=onepage&q=Hansson,%20S.%20O.%20(1996).%20Defining%20Pseudoscience.&f=false

NCS (2018) Code of Ethical Practice [Online]
Available at: https://www.nationalcounsellingsociety.org/about-us/code-of-ethics
Audio Copy: https://youtu.be/zSjS0B7-sao
Copies in braille or large format available as required: https://www.nationalcounsellingsociety.org/about-us/equal-opportunities/accessibility/

Popper, K. R. (1959) The Logic of Scientific Discovery. New York: Routledge

PsychologyToday article (2016) The “Is Psychology a Science?” Debate [Online]
Available at:https://www.psychologytoday.com/blog/theory-knowledge/201601/the-is-psychology-science-debate

UKCP (2017). Memorandum of Understanding (MoU) on Conversion Therapy in the UK. [Online]
Available at: https://www.psychotherapy.org.uk/about-ukcp/public-policy/conversion-therapy
PDF: http://www.psychotherapy.org.uk/wp-content/uploads/2017/10/UKCP-Memorandum-of-Understanding-on-Conversion-Therapy-in-the-UK.pdf

$\endgroup$
  • $\begingroup$ I appreciate such a long answer but I think you mistake my meaning somewhat. By replicability I simply mean failure of many famous studies to be reproduced. The reasons as I understand are many: publication bias, publish-or-perish culture, journals preferring new results to replications and positive results to negative, flawed statistical methods, p-hacking, underpowered studies, etc. I wouldn't say this makes these effects pseudoscience though, just inadequately studied. Anyway, given this state of affairs, I'm simply after a resource which tracks the successful/failed replications. $\endgroup$ – Philo Jun 19 '18 at 12:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.