Do personality survey responses correlate to the success or failure of minimum wage employees?

Question

I recently watched a friend fill out a personality survey for a minimum wage job. These surveys generally have a few dozen to as many as a couple hundred statements with which the applicant must agree, disagree, or stay neutral. The survey is designed to filter candidates. What puzzles me is what basis they use for this filtering.

Some of the questions are obvious correlates with negative employee qualities:

• Sometimes I shout at customers to get my point across.

Well, gee, I wonder if I should "strongly agree" with that one? Others, however, are not so clear:

• I will work hard to get a job as an executive at a large company.

• My former bosses all say I'm the best employee they've ever had.

• I spend a lot of time thinking through about a problem before solving it.

For example, being a self-starter might help or hinder a job behind a cash register. Exercising leadership is great until you're one of 40 people restocking the shelves. Thinking through problems is a useful skill unless the problem facing you is trivial, in which case "consideration" might be construed as "procrastination". Based on examples like these, it's not inherently obvious if these questions measure positive or negative qualities, or what the model response might be.

Moreover, the size, type, and prevalence of these questionnaires suggests a level of sophistication in the selection process that I'm not convinced it is due. Supposing they test 5-10 core competencies, qualities like leadership, professionalism, and decorum, there are more than enough responses to establish inter-survey response validity. Of course, that assumes these employers spent millions of dollars developing valid, proprietary psychometrical measures, or co-opted something in the academic domain.

So what are these surveys really testing? Tongue-in-cheek, if faced with such a test, what would agreeing, disagreeing, or remaining neutral with the statement, "Employer personality tests correlate with success or failure in the workplace?" reflect about an individual?

Answer 1

Do personality tests predict job performance in general?

There is a large academic literature correlating personality test scores with job performance. You might want to check out the meta-analysis by Barrick et al (2001). It reports the meta-analytic correlations often based on hundreds of studies between Big 5 personality test scores and job performance.

In general the correlations reported in the meta analysis are relatively small (e.g., $r=.12$ for conscientiousness is the biggest). These correlations get a little bigger if you adjust for measurement error in job performance. So, in summary, in general Big 5 measures of personality tend to predict job performance; however, the predictive validity is not that high. To see this in the context of other selection tests check out the meta-analysis by Schmitt and Hunter (1998).

Do personality tests predict job performance for minimum wage jobs?

I'm not sure if the category "minimum wage" necessarily captures the right distinction given such a category could refer to a diverse range of jobs.

That said, in terms of breaking analyses down by occupation, at least at this very broad meta-analytic level, the correlation between conscientiousness and job performance appeared to be fairly similar across sales performance ($r=.11$), managerial performance ($r=.12$), and skilled or semi-skilled ($r=.12$) (Barrick et al 2001).

Thus, at least from this meta-analysis, these measures of personality appear to predict job performance about as well for minimum wage jobs as they do for other jobs.

So what are these surveys really testing?

Tests can be developed in a variety of ways. In general, there are theory driven processes and there are empirically driven processes of test development. From a theory driven perspective, constructs of interest can be defined, and items can be developed. From an empirically driven perspective, various statistical techniques can be applied to check for test properties such as the degree to which items intercorrelate in meaningful ways.

Particularly in the applied setting, items or scales can be selected based on the degree to which they provide predictive validity (e.g., correlations with job performance).

More generally, personality tests provide a sampling of behaviour in a standardised way. If items have been developed in a more empirical way, then an explicit theory may not necessarily exist for the meaning of every item. Rather, it is the scale as a whole that takes on meaning which can be informed by a wide range of information including correlations within the test and correlations between the scales of the test and other measures of interest.

References

• Barrick, M.R., Mount, M.K. & Judge, T.A. (2001). Personality and performance at the beginning of the new millennium: What do we know and where do we go next?. International Journal of Selection and Assessment, 9, 9-30. PDF
• Schmidt, F.L. & Hunter, J.E. (1998). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings.. Psychological Bulletin; Psychological Bulletin, 124, 262. PDF