When is it defendable to use manifest instead of latent variables in psychological research?

Question

In psychology, most of the concepts, theories, ideas etc. of interest refer to latent variables, where a latent variable is one that can't be observed directly. Instead, one has to find a manifest variable, which can be observed directly and which can be used as an indicator for the latent variable. Unless there is a perfect indicator (which usually there is not), the measurement of the latent variable contains an error. To estimate the error, latent variable modelling techniques like CFA or Item Response Theory (among others) have been developed.

Latent Variable Theory

Now, it is generally agreed upon that variables like age and gender are manifest. But what is it that makes the difference between a latent and a manifest variable? Borsboom (2008) explicates this difference. In his latent variable theory, the distinction depends on the certainty, with which an inference can be made from the observed data to the variable in question. In this sense, a variable like gender is manifest, because one can infer with certainty if a subject is male or female from his or her answer on a questionnaire. Note that this is possible, even though one has not actually observed the gender of that subject. Borsboom's account of latent variable theory also implies that the status of a variable as latent vs. manifest can change over time, if for example more information becomes accessible. But as long as a variable has proven to be manifest, it remains latent.

It seems to me, then, that pretty much every researcher in psychology should use latent variable modelling techniques, since most of the time it is not at all possible to infer something with certainty. However, this is clearly not the case. There are numerous published articles that use manifest variables and analyze the relationships between them without accounting for measurement error, thereby implicitly making the latent variable in question a manifest variable. (If anyone wants citations for this claim, I will happily provide them.)

Question: Under what circumstances is it defendable to use manifest instead of latent variables in psychological research, even if it is clear that the variables can only be measured with error?

Borsboom, D. (2008). Latent variable theory. Measurement: Interdisciplinary Research & Perspective, 6(1-2), 25-53. PDF

Answer 1

Experimental psychologists seem quite happy working with single item ad hoc self-report scales, physiological measures, etc. with very little psychometrical assessment so even before talking about a full-fledged latent variable modeling approach, confirmatory factor analysis and the like you might want to wonder why they appear relatively unconcerned with measurement issues in general and why there is still such a big gap between experimental psychology and psychometrics.

Some potential explanations (some of them are not very convincing justifications for not doing latent variable modeling but still explain why people are content using manifest variables):

• Experimentalist can create (relatively) strong effects. If you are doing an experiment in which you try to induce disgust by showing pictures of disgusting things to people and your pictures do not seem to produce a measurable effect, you could try to improve your measurement but you can also just use even more disgusting pictures or add a few additional trials in your experiment. On the other hand, when you are creating a personality test for recruitment purposes, you can't just wish candidates would be more different from each other to make your task easier or be happy with estimating the average of all candidates, you really need to assign a score to each candidate with a level of precision that allows you to discriminate between them.
• They rely on the nature of the manipulation and its effects on the response and not mainly on the correlation between different scales to interpret their results.
• They are often interested in group differences. (Inter-individual differences often dwarf any item-related error so why even care about that? Just add a few participants!)
• They still structure their experiments as a test of some ‘nil’ hypothesis (is there an effect or not?)
• They are not very interested in effect sizes as such, a consequence of the previous point (if they do compute some effect size measure, it's mostly with an eye for statistical power).

In short, if any statistically significant difference is regarded as an interesting result (i.e. your manipulation had an effect), you don't need to be too concerned about the reliability or meaning of the response.

Interestingly, you seem to be approaching the question from one camp, assuming that latent variable modeling is clearly useful and people need to actively defend themselves for not doing the obvious right thing. You might also turn the question around and ask “Is latent variable modeling all that useful? What does it buy us?” In the context I just highlighted, it might not be so easy to articulate a convincing answer.

And of course, in practice, many people just care about methodology to the smallest extent possible to go about their business, be published and do the things they regard as really interesting. Consequently, many researchers don't hear much about latent variable modeling after they finished their master and they don't even care about it one way or the other because it's just not the way things are done in their disciplines.

Answer 2

It's an interesting question. Here are a bunch of thoughts that came to my mind for why researchers might focus on observed variables.

• Many researchers report reliability and observed relationships between variables. By adopting a few assumptions, the reader can estimate what the latent relationships would be (see for example, the formula for correction of attenuation).

• Some researchers interpret their effect sizes relative to other studies that have reported observed variables. Thus, effect sizes are not interpreted in absolute terms. In areas where effect sizes are typically reported using observed variables, this frame of reference may make most sense. That said, this becomes problematic where reliability is varying between studies. In these cases, Hunter and Schmidt style meta-analytic corrections for unreliability generally lead to more comparable results.

• Modelling observed variables involves fewer modelling choices than modelling latent variables. In particular, estimates of relationships between latent variables are contingent on the assumptions for estimating latent variables. If, as a reader, you disagree with these assumptions, it can be difficult to recover the relationships between the observed variables.

• Modelling observed variables is simpler than modelling latent variables. This helps to explain why many researchers report correlation matrices and regressions in psychology, rather than latent variable equivalents. However, things can get much more complex as you step outside the standard multivariate normal SEM contexts: e.g., incorporating categorical variables, moderator effects, latent variables that are ill-defined, non-linear effects, non-normal error variance, correlated error variance, and so on. It's understandable that some social science researchers fall back on to the more familiar territory of modelling observed variables in a piece-wise fashion. However, while this is an explanation, researchers should still strive to model data in more integrated and realistic ways.

• Ultimately, it is the researcher who decides whether they wish to draw inferences about relationships between observed or latent variables. While some contexts suggest that the researcher should be interested in the underlying construct, it is still up to the researcher to decide.

• Relationships between observed variables are sometimes primarily of interest. Some examples: (a) experimental conditions; (b) when researchers are interested in applied prediction; (c) when researchers are interested in the observed variable itself.

• Whether researchers model latent variables seems to be related to the domain. In particular, modelling latent variables seems to be particularly popular in fields with large samples, observational designs, that use self-report or multiple choice psychological scales. In such contexts the assumption that the observed variables are random sample of possible observed variables and are a manifestation of a latent variable often seems more appropriate. It's easy to imagine the alternative items that could have been collected, and how the reliability could have been improved had more items been included. It is interesting to ponder why this is the case, and whether there are areas that would benefit from greater incorporation of latent variable modelling.

• Often the latent variable of interest is not the variable implied by the intercorrelation of observed variables. For example, the latent factor underlying a set of self-report extraversion items will not be true extraversion. I can conceptualise a true extraversion that might be calculated were it possible to integrate everything that could be known about a person's behaviour, their thoughts, their physiology, and their environmental regularities. However, the factor underlying the intercorrelations of self-report items would not be this factor. While there is value to estimating relationships had perfect reliability been obtained, this should not absolve us from trying to estimate the actual latent variable of interest.

• Correcting for measurement error should not stop researchers from trying to increase the reliability and validity of measurement. Correcting for unreliability is not as good as measuring reliably in the first place.

Answer 3

The meaning of latent variables have the unfortunate downside of changing between samples because they are fit to the observed correlation matrix (likely only jittering the meaning, but definitely changing from sample to sample).

With this, I would argue that it is best to use manifest variables when using a standard scale that you intend to compare your results to others using that standard scale.

The other reason to use manifest variables is if there are established cut points that you want to use.

I would use a latent variable for anything home brewed or for any construct where I collected multiple scales/other indicators on.