Is it correct to say that, when making inferrence on the results of an experiment, using a random-effects (RFX) rather than a fixed-effects (FFX) model merely makes the results more generalisable, as the quote below (from the fMRI study by Leaver et al., 2009) seems to suggest?

Random-effects models attempt to remove intersubject variability, thereby making the results obtained from a limited sample more generalizable to the entire population (Petersson et al., 1999). Thus, we used random-effects models in our analyses. In a single instance, we additionally considered results of a fixed-effects model, in which intersubject variability is not removed.

These authors later make the claim that those brain areas identified with the RFX analysis are more reliably/consistently involved.

I'd have thought that, with FFX, it is not the case that the results are simply "less" generalisable, but instead that they simply do NOT allow generalisation in quite the same way, since you are not accounting for the sampling noise added by the random selection of subjects to the sample. In other words, that there is a step jump rather than a gradual increase in reliability.

Furthemore, in my mind, this sounds like a fallacy somehow comparable to making inferrence on p-values that were not corrected for multiple comparisons: it's not that the corrected values are "more reliable" against false positives, it's simply that the uncorrected ones are UNreliable.

Does this analogy (or some other one) apply here, when using FFX (and not RFX) for sample-to-population inference?

  • 2
    $\begingroup$ Sounds like a question for stats.stackexchange.com $\endgroup$ – Fizz Jun 27 '18 at 15:26
  • $\begingroup$ Certainly does - one that nonetheless sadly received no answers. $\endgroup$ – z8080 Jun 28 '18 at 8:11
  • $\begingroup$ @z8080 Is the question at stats still there? If so, could you link, please? $\endgroup$ – Steven Jeuris Jun 28 '18 at 17:56
  • $\begingroup$ Hi Steven, the link is in my previous comment, here it is again: stats.stackexchange.com/questions/348722/… $\endgroup$ – z8080 Jun 29 '18 at 7:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.