Consider for exemple the stereotype threat, of which a recent meta-analysis can be found here. People try to assess how can a stereotype might explain a difference between two groups.

However, they almost never control for IQ or Big Five which seems to me to be latent variables with much more explanatory power.

My questions are :

  1. Why don't they ever control for these two important factors ?
  2. Does it mean that the conclusion of these studies are flawed in some way ?
  • $\begingroup$ Genetics are another major explanatory factor for many psychological constructs, including IQ and personality. Just think practically what it would take in terms of power, to control for such factors - studies would need orders of magnitude more participants. $\endgroup$
    – Arnon Weinberg
    Nov 17, 2020 at 20:26
  • 2
    $\begingroup$ Yes I understand that it would greatly reduce the power of the studies but wouldn't it be better to accept null hypothesis because of a too small sample rather than derive conclusions from flawed premisses ? $\endgroup$ Nov 18, 2020 at 13:35
  • $\begingroup$ I always wondered about the opposite. Why would researchers control for all sorts of variables although their theories and hypotheses do not say anything about these variables. Including variables without logic has little to do with the scientific method. $\endgroup$ Aug 23, 2021 at 11:51

2 Answers 2


A quick look at your reference focussed on a meta-analysis of an intervention. Evaluating interventions is mostly about estimating the mean effect of an intervention. So, even if personality or intelligence influences the outcome, this is not directly relevant to evaluating whether on average the intervention has an effect.

In other contexts, controlling for personality and intelligence maybe quite relevant. In general, controlling for personality or intelligence is more relevant (a) where the predictors are conceptually or empirically related to personality or intelligence, and (b) where personality or intelligence is a strong predictor of the outcome of interest.

There are a whole range of psychological constructs that overlap substantially with either personality or intelligence. And in those cases, you do have a body of literature that examines the degree of overlap with personality or intelligence. And there's also literature that examines the incremental prediction of the variable of interest relative to personality.

For example:

  • Grit: Is it different from conscientiousness?
  • Assessment centers: Do they incrementally predict over and above intelligence?
  • Type D personality: Is it more than a composite of neurotcism and extraversion
  • Trait emotional intelligence: Is it just a weighted composite of the Big 5
  • Ability emotional intelligence: does it predict workplace outcomes over and above intelligence?

Researchers are always confronted with issues of which variables to measures. It is often not practical to measure everything. In particular, intelligence takes quite a bit of time to measure well. So you often get research that doesn't control for these factors. And then later on, other studies come along and fill the gap. There is also a bit of a process "construct proliferation" in psychology. A researcher comes up with a new pet construct. And other researchers point out later on how it's not that different to personality or how it doesn't incrementally predict. A lot of heated debates arise over these things.

Researchers also differ in theoretical orientation and practical concerns. For instance, in industrial/organisational psychology, intelligence and Big 5 personality assessment are very well embedded. So, incremental prediction over and above these factors is a common question for a researchers to ask. In other fields, they may be less grounded in personality or intelligence frameworks.

  • 2
    $\begingroup$ Ideally, in a randomized experiment there should be no differences in your groups in other factors - that's why you randomize. You could still investigate variables that affect the strength of the intervention, but to evaluate the intervention in general it's not necessary. Not sure if the interventions OP referred to involved randomized experiments. $\endgroup$
    – Bryan Krause
    Nov 18, 2020 at 1:47
  • 2
    $\begingroup$ Hi Bryan. I agree with all your points. $\endgroup$ Nov 18, 2020 at 2:27
  • $\begingroup$ @BryanKrause this is the point I'm struggling with, as a student in statistics, I expect some difference between the groups even if they are sampled from the same population, just because of randomness so I wonder if controlling for iq between the measuring and the control group might not improve our interpretations of the psychological litterature. $\endgroup$ Nov 21, 2020 at 11:30
  • $\begingroup$ @JeandeLéry What about all the other possible confounders you fail to measure? In a small trial, you may have imbalances that affect the result, but not enough data to effectively correct for it. In a larger trial it becomes exceedingly unlikely to find large imbalances on any one variable. Same for meta analyses. Unless there is a bias where a confounder makes people drop out of one arm but not the other, randomization solves the issue. $\endgroup$
    – Bryan Krause
    Nov 21, 2020 at 13:11
  • $\begingroup$ I'm no psychology graduate, let alone expert so I thought that IQ and big five were standards like weight and height in biometrics and thus it would make sense to control for these two as when I am doing a linear model in biology I always control for weight & height. But then, if actually there are much more significant cofounders of a size comparable to big 5 and IQ, then my idea was flawed and I am glad you corrected it $\endgroup$ Nov 21, 2020 at 14:32

You only control for factors that have been proven to matter. IQ has not been shown to be significant. So why measure or control IQ variation if there is no evidence of difference.

In fact there is evidence that the activation of stereotype threat confounds cognitive performance and therefore IQ. If a person has their stereotype threat activated, and they perform poorly in an IQ test, is that a fair representation of their cognitive abilities? Many articles and studies prove the confounding ability of stereotype threat and the need to minimise this factor in IQ testing, it seems complex and unfair to apply a measure that is significantly influenced by the stereotype threat factor that you are studying.

- Stereotypes impair intellectual performance blog

  • $\begingroup$ Comments are not for extended discussion; this conversation has been moved to chat. $\endgroup$
    – Arnon Weinberg
    Aug 2, 2022 at 16:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.