What measure or criteria wording can be used to track trends in p factor?

Question

Background: you can use factor analysis on mental illness to get a general psychopathology factor, like the g factor in intelligence but instead a general mental illness factor.

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4209412/

Intent: I am interested in using a forecast aggregation platform to predict whether we will discover a Flynn effect like trend in p factor, either up or down. https://www.metaculus.com/questions/

Caveat: Flynn may be hollow for g, and in theory something like that could happen with a p factor trend too. That's fine. Our forecast question can be agnostic about the nature of the trend, in order to be reliably resolvable without too much administrative effort.

Specific need: what simple metric, or criteria wording, could be shown to the forecasters, such that they could predict something that will reliably resolve? In forecasting, the event description must be unambiguous. This is harder than people new to forecasting tend to expect. One thing that might affect our criterion choice is how we think future researchers will measure such a thing, as piggybacking off already-reported metrics can make forecast question-writing easier.

E.g. suppose if we had the following wording: "Will there be data to support a trend in the p factor, at least equivalent to an average 2 IQ point-per-decade change, either up or down, during any 10-year period before 2040?" Would it be easy for people not working in any related field to kind of just, do a cursory search around 2040 (or every few years) and clearly tell if such a thing has happened? If not, what would be a better way to word it?

Any input appreciated!

Answer 1

Bottom line: I think what you are interested in is trends in psychopathology, rather than p-factor.

The terms "g" and "g-factor" are actually 2 constructs often conflated in the literature. One way that they are used is to refer to a shared variance or correlation coefficient between intelligence sub-tests, calculated at a population level using a factor analysis, to produce a value between 0 and 1. This definition is more precisely referred to as "psychometric g" or "Spearman's g". Another way that they are used is to refer to a general underlying (physical but unknown) mechanism through which intelligence is generalized in the brain, measured at an individual level using IQ tests, with values averaging 100 and a standard deviation of 15. This definition may be referred to as general intelligence or IQ. Currently, work on p-factor mostly refers to a construct analogous to a "psychometric p", whereas easily determining trends requires analysis of general psychopathology testing analogous to IQ.

Unambiguously defining a forecasting question on secular trends in psychopathology has a few requirements:

• Qualified tests: IQ actually refers to a variety of different tests that either label themselves as IQ tests, or are generally considered as such. There are numerous general psychopathology tests, such as SCL-90, MINI, AUDADIS, SBQ, and many more (see Constantinou & Fonagy, 2019 - Appendix A for a sample) that have been used in p-factor research. Unfortunately, the tests do not self-label, nor is a comprehensive list available anywhere that I can find (though try Wikipedia for a starting point).

• Data pool: Research on cross-cohort trends using these tests has been done in the past - examples: Collishaw (2015), Bor et al (2014), Schürmann & Margraf (2018), Kronström et al (2021). Note that thus far such research has been sporadic, and limited to specific populations. A relatively good example is Twenge et al (2010) which uses the MMPI, and even reports results in terms of standard deviation:

... among American college students (N=63,706) between 1938 and 2007 on the MMPI and MMPI-2 and high school students(N=13,870) between 1951 and 2002 on the MMPI-A. The current generation of young people scores about a standard deviation higher (averaged=1.05) on the clinical scales ...

• Normalized results: The below image from Caspi et al (2014) shows what psychopathology test results look like when normalized - notice the similarity to IQ test results. However, most psychopathology test results are not normalized in this manner, making them challenging to compare.

• Stable p-loadings: Just as IQ trends may be hollow for g, psychopathology trends may be hollow for p. Accordingly, if you wanted trends in actual p-factor, then cross-cohort studies testing it for stable shared variances or loadings would be needed as well (analogous to the Jensen effect in IQ). Currently, the p-factor construct faces much criticism and has yet to prove itself tenable. This is not an issue if you are willing to be agnostic about p.

Given all these caveats, you may phrase a forecasting question along the lines of: "Using results from general psychopathology tests, measuring p-factor at an individual level, will there be data to support a trend ..." Resolution of this question would be dependent on such tests being sufficiently identifiable, popularized, and normalized at the time of the beginning of the predicted trend.