Richard Griffith and Patrick Converse (2012) provide a good review of personality faking research that has employed a within-subjects design to examine changes in responses to personality tests between an applicant and a development context.
The following represent my review of the papers identified by Griffith and Converse (2012). I take it as fairly well established in the literature that most, but not all, studies find differences in group means between between different applicant and non-applicant conditions. This review focusses on individual differences in whether people fake and the amount of faking.
Arthur et al (2010)
Among other things, Arthur et al (2010) reported two studies that looked at differences in responding between participants completing a personality test as job applicant and then around a year later completing the same personality test as an anonymous research participant.
SEM_d: The authors used the standard error of test-retest measurement $\text{SEM}_d$ to classify in an operationalised way whether a participant had engaged in response distortion in the high-stakes context
- $\text{SEM}_d \lt 1$ indicated lower score at time 2 (i.e., faking)
- $\text{SEM}_d \gt 1$ indicated higher score
- $|\text{SEM}_d| \lt 1$ indicated unchanged score (i.e., honest)
Results in study 1 suggested that for most personality scales a little over 30% of scores were classified as lower in the low stakes condition.
Of course this value should be interpreted in light of the fact that based on the normal distribution and the chosen cut-offs, one would expect around 16% above this cut-off. There are also issues of regression to the mean, and reduced reliability in the extremes that could be considered in a more sophisticated analysis. Finally, this indicator of faking does not provide information on the distribution of faking scores.
SEM_d by job applicant score:
They also explore the prevalence of score reduction relative to scores in the high stakes condition. They conclude that prevalence of score reduction does not vary with High stakes scores. However, the graphs are a little difficult to interpret (being stacked bar charts). Also, by design the graphs answer the question of what proportion of test takers at a given level are faking, which is interesting , but they don't answer the question of whether true score is related to faking.
Correlation between applicant and research context scores:
Table 5 reports $\text{time}_1 - \text{time}_2$ correlations for both studies for the Big 5. Correlations are between .49 and .78.
Such correlations make sense in light of the idea that some participants are answering honestly and others are faking. This should produce correlations a little lower than typical test-retest reliabilities.
Of course, many factors influence correlations of personality test scores in applicant and research contexts. In particular, reliability of the test and time between tests would be important. Thus, it's a little difficult to say how much of the observed reduction in the correlation is due to faking in the applicant context, and how much is due to other factors.
The researchers do provide an $n=44$ sample of participants in Study 1 who completed both conditions in low stakes contexts (i.e., existing employee and research contexts). These correlations are reported in Table 6. The following analysis using R shows the correlations for the two conditions, where theory would suggest that the High Stakes-Low Stakes (i.e., HL) condition should have smaller correlations, although the observed difference is fairly minimal.
x <- data.frame(matrix(c(0.53, 0.55, 0.51, 0.7, 0.66, 0.56, 0.49,
0.5, 0.87, 0.75), ncol = 2))
names(x) <- c("HL", "LL")
row.names(x) <- c("Agreeableness", "Conscientiousness", "Emotional stability",
"Extraversion", "Openness")
x$diff <- x$HL - x$LL
round(rbind(x, Mean = sapply(x, mean)), 2)
## HL LL diff
## Agreeableness 0.53 0.56 -0.03
## Conscientiousness 0.55 0.49 0.06
## Emotional stability 0.51 0.50 0.01
## Extraversion 0.70 0.87 -0.17
## Openness 0.66 0.75 -0.09
## Mean 0.59 0.63 -0.04
Griffith et al (2007)
Griffith et al (2007) conducted a study where 60 applicants completed a 30-item measure of conscientiousness first in a job applicant setting. Then one month later, they completed the test under instructions to answer the test honestly and to fake.
They found a correlation between honest and applicant of $r =.50$, and a standardised difference between honest and applicant of $d = 61$.
This shows that scores are increasing in applicant settings, and that there are individual differences in the extent to which this occurs. Although the absence of knowledge of the test-retest correlation in an honest-honest context makes it a little difficult to assess the degree to which this correlation is less than would be expected.
They also employed several indices to act as difference score thresholds, which classified 31 or 22 percent of applicants as faking depending on the index.
In a latter part of the paper there is also some discussion of variation in the size of the difference score.
Ellingson et al (2007):
Ellinson et al performed a study based on a large database of participants who had completed a personality test under the four combinations of development and selection contexts.
Average Development-development sample correlations were $.66$ were a little bit higher than average development-selection ($r=.62$) and selection-development ($r=.57$) correlations.
Average increases from session 1 to 2 suggested somewhat low levels of response distortion.
average scale d
development/development 0.14
development/selection 0.26
selection/development 0.04
selection/development 0.07
As far as I can tell, there was no specific discussion of the distribution of change scores.
Others
Donovan et al (2008) and Peterson et al (2008) sound interesting, but they are conference presentations so the publications are not readily available.
Summary
Thus, in summary, there is some evidence that the correlation is lower between applicant and development conditions than it would between two comparable development conditions.
Presumably the size of this drop in correlation is related to the amount of faking, and that the amount of faking is related to the amount of individual differences in faking.
There is certainly a qualitative awareness of there being different categories of faking.
Ellingson write:
Conservative estimates from this line of research suggest that
less than 20% of applicants intentionally distort their responses
and that less than 4% of applicants do so to an extreme degree
However, I haven't yet read much from these within-subject studies about the distribution of faking scores, nor have I seen formal mathematical models of the change scores, even though the data would permit such analyses.
References
- Arthur, W., Jr., Glaze, R. M., Villado, A. J., & Taylor, J. E. (2010). The magnitude and extent of cheating and response distortion effects on unproctored Internet based test of cognitive ability and personality. International Journal for Selection and Assessment, 18, 1-16. PDF
- Donovan, J. J., Dwight, S. A., & Schneider D. (2008). Faking in the real world: Evidence from a field study. In R. L. Griffith & M. H. Peterson (Chairs), Examining faking using within-subjects designs and applicant data. Symposium conducted at the 23rd Annual Conference for the Society for Industrial and Organisational Psyhcology, San Francisco, CA.
- Ellingson, J. E., Sackett, P. R., & Connelly, B. S. (2007). Personality assessment across selection and development contexts: Insights into response distortion. Journal of Applied Psychology, 92, 386-395.
- Griffith, R. L. & Converse, P. D. (2012). The rules of evidence and the prevalence of applicant faking in New Perspectives on Faking in Personality Assessment, Chapter 3, 34-52.
- Griffith, R. L., Chmielowski, R., & Yoshita, Y. (2007). Do applicants fake? An examination of the frequency of applicant faking behavior. Personnel Review, 36, 341-355. PDF
- Peterson, M. P. Griffith, R. L., O'Connel, M. S., & Isaacson, J. A. (2008). Examining faking in real job applicants: A with-subjects investigation of score changes across applicant and research settings. Paper presented at the 23rd annual meeting for the Society for Industrial and Organizational Psychology, San Francisco, California.
