Return to Answer

Experimental psychologists seem quite happy working with single item ad hoc self-report scales, physiological measures, etc. with very little psychometrical assessment so even before talking about a full-fledged latent variable modeling approach, confirmatory factor analysis and the like you might want to wonder why they appear relatively unconcerned with measurement issues in general and why there is still such a big gap between experimental psychology and psychometrics.

Some potential explanations (some of them are not very convincing justifications for not doing latent variable modeling but still explain why people are content using manifest variables):

• Experimentalist can create (relatively) strong effects. If you are doing an experiment in which you try to induce disgust by showing pictures of disgusting things to people and your pictures do not seem to produce a measurable effect, you could try to improve your measurement but you can also just use even more disgusting pictures or add a few additional trials in your experiment. On the other hand, when you are creating a personality test for recruitment purposes, you can't just wish candidates would be more different from each other to make your task easier or be happy with estimating the average of all candidates, you really need to assign a score to each candidate with a level of precision that allows you to discriminate between them.
• They rely on the nature of the manipulation and its effects on the response and not mainly on the correlation between different scales to interpret their results.
• They are often interested in group differences. (Inter-individual differences often dwarf any item-related error so why even care about that? Just add a few participants!)
• They still structure their experiments as a test of some ‘nil’ hypothesis (is there an effect or not?)
• They are not very interested in effect sizes as such, a consequence of the previous point (if they do compute some effect size measure, it's mostly with an eye for statistical power).

In short, if any statistically significant difference is regarded as an interesting result (i.e. your manipulation had an effect), you don't need to be too concerned about the reliability or meaning of the response.

Interestingly, you seem to be approaching the question from one camp, assuming that latent variable modeling is clearly useful and people need to actively defend themselves for not doing the obvious right thing. You might also turn the question around and ask “Is latent variable modeling all that useful? What does it buy us?”

And of course, in practice, many people just care about methodology to the smallest extent possible to go about their business, be published and do the things they regard as really interesting. Consequently, many researchers don't hear much about latent variable modeling after they finished their master and they don't even care about it one way or another because it's just not the way things are done in their disciplines.

Experimental psychologists seem quite happy working with single item ad hoc self-report scales, physiological measures, etc. with very little psychometrical assessment so even before talking about a full-fledged latent variable modeling approach, confirmatory factor analysis and the like you might want to wonder why they appear relatively unconcerned with measurement issues in general and why there is still such a big gap between experimental psychology and psychometrics.

Some potential explanations (some of them are not very convincing justifications for not doing latent variable modeling but still explain why people are content using manifest variables):

• Experimentalist can create (relatively) strong effects. If you are doing an experiment in which you try to induce disgust by showing pictures of disgusting things to people and your pictures do not seem to produce a measurable effect, you could try to improve your measurement but you can also just use even more disgusting pictures or add a few additional trials in your experiment. On the other hand, when you are creating a personality test for recruitment purposes, you can't just wish candidates would be more different from each other to make your task easier or be happy with estimating the average of all candidates, you really need to assign a score to each candidate with a level of precision that allows you to discriminate between them.
• They rely on the nature of the manipulation and its effects on the response and not mainly on the correlation between different scales to interpret their results.
• They are often interested in group differences. (Inter-individual differences often dwarf any item-related error so why even care about that? Just add a few participants!)
• They still structure their experiments as a test of some ‘nil’ hypothesis (is there an effect or not?)
• They are not very interested in effect sizes as such, a consequence of the previous point (if they do compute some effect size measure, it's mostly with an eye for statistical power).

In short, if any statistically significant difference is regarded as an interesting result (i.e. your manipulation had an effect), you don't need to be too concerned about the reliability or meaning of the response.

Interestingly, you seem to be approaching the question from one camp, assuming that latent variable modeling is clearly useful and people need to actively defend themselves for not doing the obvious right thing. You might also turn the question around and ask “Is latent variable modeling all that useful? What does it buy us?” In the context I just highlighted, it might not be so easy to articulate a convincing answer.

And of course, in practice, many people just care about methodology to the smallest extent possible to go about their business, be published and do the things they regard as really interesting. Consequently, many researchers don't hear much about latent variable modeling after they finished their master and they don't even care about it one way or the other because it's just not the way things are done in their disciplines.

Experimental psychologists seem quite happy working with single item ad hoc self-report scales, physiological measures, etc. with very little psychometrical assessment so even before talking about a full-fledged latent variable modeling approach, confirmatory factor analysis and the like you might want to wonder why they appear relatively unconcerned with measurement issues in general and why there is still such a big gap between experimental psychology and psychometrics.

Some potential explanations (they are not necessarily very convincing justifications):

• Experimentalist can create (relatively) strong effects. If you are doing an experiment in which you try to induce disgust by showing pictures of disgusting things to people and your pictures do not seem to produce a measurable effect, you could try to improve your measurement but you can also just use even more disgusting pictures or add a few additional trials in your experiment. On the other hand, when you are creating a personality test for recruitment purposes, you can't just wish candidates would be more different from each other to make your task easier or be happy with estimating the average of all candidates.
• They rely on the nature of the manipulation and its effects on the response and not mainly on the correlation between different scales to interpret their results.
• They are often interested in group differences (inter-individual differences often dwarf any item-related error so why even care about that, just add a few participants)
• They still structure their experiments as a test of some ‘nil’ hypothesis (is there an effect or not?)
• They are not very interested in effect sizes as such, a consequence of the previous point (if they do compute some effect size measure, it's mostly with an eye for statistical power).

In short, if any statistically significant difference is regarded as an interesting result (i.e. your manipulation had an effect), you don't need to be too concerned about the reliability or meaning of the response.

Interestingly, you seem to be approaching the question from one camp, assuming that latent variable modeling is clearly useful and people need to actively defend themselves for not doing the obvious right thing. You might also turn the question around and ask “Is latent variable modeling all that useful? What does it buy us?” In practice, many people don't even care about latent variable modeling one or another because it's just not the way things are done in their disciplines.

Experimental psychologists seem quite happy working with single item ad hoc self-report scales, physiological measures, etc. with very little psychometrical assessment so even before talking about a full-fledged latent variable modeling approach, confirmatory factor analysis and the like you might want to wonder why they appear relatively unconcerned with measurement issues in general and why there is still such a big gap between experimental psychology and psychometrics.

Some potential explanations (some of them are not very convincing justifications for not doing latent variable modeling but still explain why people are content using manifest variables):

• Experimentalist can create (relatively) strong effects. If you are doing an experiment in which you try to induce disgust by showing pictures of disgusting things to people and your pictures do not seem to produce a measurable effect, you could try to improve your measurement but you can also just use even more disgusting pictures or add a few additional trials in your experiment. On the other hand, when you are creating a personality test for recruitment purposes, you can't just wish candidates would be more different from each other to make your task easier or be happy with estimating the average of all candidates, you really need to assign a score to each candidate with a level of precision that allows you to discriminate between them.
• They rely on the nature of the manipulation and its effects on the response and not mainly on the correlation between different scales to interpret their results.
• They are often interested in group differences. (Inter-individual differences often dwarf any item-related error so why even care about that? Just add a few participants!)
• They still structure their experiments as a test of some ‘nil’ hypothesis (is there an effect or not?)
• They are not very interested in effect sizes as such, a consequence of the previous point (if they do compute some effect size measure, it's mostly with an eye for statistical power).

In short, if any statistically significant difference is regarded as an interesting result (i.e. your manipulation had an effect), you don't need to be too concerned about the reliability or meaning of the response.

Interestingly, you seem to be approaching the question from one camp, assuming that latent variable modeling is clearly useful and people need to actively defend themselves for not doing the obvious right thing. You might also turn the question around and ask “Is latent variable modeling all that useful? What does it buy us?”

And of course, in practice, many people just care about methodology to the smallest extent possible to go about their business, be published and do the things they regard as really interesting. Consequently, many researchers don't hear much about latent variable modeling after they finished their master and they don't even care about it one way or another because it's just not the way things are done in their disciplines.

Experimental psychologists seem quite happy working with single item ad hoc self-report scales, physiological measures, etc. with very little psychometrical assessment so even before talking about a full-fledged latent variable modeling approach, confirmatory factor analysis and the like you might want to wonder why they appear relatively unconcerned by measurement issues and why there is still such a big gap between experimental psychology and psychometrics.

Some potential explanations (they are not necessarily very convincing justifications):

• Experimentalist can create (relatively) strong effects. If you are doing an experiment in which you try to induce disgust by showing pictures of disgusting things to people and your pictures do not seem to produce a measurable effect, you could try to improve your measure of disgust but you can also just use even more disgusting pictures or add a few additional trials in your experiment.
• They rely on the nature of the manipulation and its effects on the response and not mainly on the correlation between different scales to interpret their results.
• They are often interested in group differences (inter-individual differences often dwarf any item-related error so why even care about that, just add a few participants)
• They still structure their experiments as a test of some ‘nil’ hypothesis (is there an effect or not?)
• They are not very interested in effect sizes as such, a consequence of the previous point (if they compute it, it's with an eye for statistical power).

In short, if any statistically significant difference is regarded as an interesting result (i.e. your manipulation had an effect), you don't need to be too concerned about the reliability or meaning of the response.

Interestingly, you seem to be approaching the question from one camp, assuming that latent variable modeling is clearly useful and people need to actively defend themselves for not doing the obvious right thing. You might also turn the question around and ask “Is latent variable modeling all that useful? What does it buy us?”

Experimental psychologists seem quite happy working with single item ad hoc self-report scales, physiological measures, etc. with very little psychometrical assessment so even before talking about a full-fledged latent variable modeling approach, confirmatory factor analysis and the like you might want to wonder why they appear relatively unconcerned with measurement issues in general and why there is still such a big gap between experimental psychology and psychometrics.

Some potential explanations (they are not necessarily very convincing justifications):

• Experimentalist can create (relatively) strong effects. If you are doing an experiment in which you try to induce disgust by showing pictures of disgusting things to people and your pictures do not seem to produce a measurable effect, you could try to improve your measurement but you can also just use even more disgusting pictures or add a few additional trials in your experiment. On the other hand, when you are creating a personality test for recruitment purposes, you can't just wish candidates would be more different from each other to make your task easier or be happy with estimating the average of all candidates.
• They rely on the nature of the manipulation and its effects on the response and not mainly on the correlation between different scales to interpret their results.
• They are often interested in group differences (inter-individual differences often dwarf any item-related error so why even care about that, just add a few participants)
• They still structure their experiments as a test of some ‘nil’ hypothesis (is there an effect or not?)
• They are not very interested in effect sizes as such, a consequence of the previous point (if they do compute some effect size measure, it's mostly with an eye for statistical power).

In short, if any statistically significant difference is regarded as an interesting result (i.e. your manipulation had an effect), you don't need to be too concerned about the reliability or meaning of the response.

Interestingly, you seem to be approaching the question from one camp, assuming that latent variable modeling is clearly useful and people need to actively defend themselves for not doing the obvious right thing. You might also turn the question around and ask “Is latent variable modeling all that useful? What does it buy us?” In practice, many people don't even care about latent variable modeling one or another because it's just not the way things are done in their disciplines.