How do personality tests work that require you to choose one of two statement that applies most to you?

Question

There are different kinds of psychometric tests, I think the numerical and verbal ones are fair enough, I can see their usefulness. However, I cannot understand how the 'soft' psychometric tests work and how they apply to a particular career.

I just finished one which consisted of 90 'questions', except they were not questions, they were all "pick one of the two following sentences" 90 times. Some made sense, such as

• Do you like to lead a group.
• Do you like to fit in with a group.

But others I could not answer because I could not choose either option for example:

• You are slow at planning.
• You are slow at making decisions.

I have to answer one. In this case, I'd say that I am not slow in either.

How exactly (detail) do these types of psychometric tests work? A link to an open one would be what I am looking for. How do they create the answers? How do they pair them? Do they change based on your previous answers?

Answer 1

The basic idea is that you get a sample population. Let's say you have 100 people. You give each of those a creativity rating through some process.

Afterwards you give every person in the sample population a stack of questions. You run statistical tests to determine which of those questions correlate with someone being creative. You keep all the questions that strongly correlate with your original creativity rating and throw questions that don't correlate away.

When making a test for a job application you also throw questions that seem to have "correct" answer and who therefore allow for cheating away. Alternatively you can also use those questions to determine the amount to which the test subject tries to game the test.

Answer 2

There is a large general literature evaluating the degree to which personality tests predict job performance. In particular see for example the review by Barrick et al (2001). In general such reviews find that personality measures provide a small but meaningful prediction of job performance.

Ipsative testing

So in general, you are asking about how personality tests work. But more specifically, you are asking about ipsative personality tests:

Ipsative is a descriptor used in psychology to indicate a specific type of measure in which respondents compare two or more desirable options and pick the one that is most preferred (sometimes called a "forced choice" scale).

Regarding your statement that you could not answer one of the questions, presumably the instructions asked you to indicate which statement applied most. Thus, it would be hoped that even if there were two statements that both applied to you a lot, or both applied a little, you would be able to indicate which applied more.

For general scientific use where you can rely on people to answer honestly, researchers typically do not use ipsative testing. Ipsative testing generally distorts the correlations between subscales.However, ipsative tests is popular in applied settings such as in selection and recruitment. The argument is that ipsative testing reduces faking.

Relevant references include:

Heggestad et al (2006) interpreted their results to indicate that ipsative testing was not a viable method for controlling for faking:

Recent research suggests multidimensional forced-choice (MFC) response formats may provide resistance to purposeful response distortion on personality assessments. It remains unclear, however, whether these formats provide normative trait information required for selection contexts. The current research evaluated score correspondences between an MFC format measure and 2 Likert-type measures in honest and instructed-faking conditions. In honest response conditions, scores from the MFC measure appeared valid indicators of normative trait standing. Under faking conditions, the MFC measure showed less score inflation than the Likert measure at the group level of analysis. In the individual-level analyses, however, the MFC measure was as affected by faking as was the Likert measure. Results suggest the MFC format is not a viable method to control faking.

However, Bowen et al reached a more positive conclusion about ipsative testing and the degree to which it is faking-resistant:

The current study extends previous research by comparing empirical data on ipsative and normative versions of the 30-scale Occupational Personality Questionnaire (OPQ). Participants were randomly assigned to “honest” or “faking” groups. Results support the proposition that when the number of scales is large, the measurement dependency problem associated with the ipsative format is negligible. There was satisfactory convergent validity between ipsative and normative forms in both groups. On both forms, the distance between an ideal profile and participants' ratings were narrower in the faking group than in the honest group. The results suggest that though ipsative measures were not completely free from faking, they were relatively more effective in guarding against faking than more commonly used normative measures.

Meade (2004) discuss ipsative measurement more generally.

Data are described as ipsative if a given set of responses always sum to the same total. However, there are many properties of data collection that can give rise to different types of ipsative data. In this study, the most common type of ipsative data used in employee selection (forced-choice ipsative data; FCID) is discussed as a special case of other types of ipsative data. Although all ipsative data contains constraints on covariance matrices (covariance-level interdependence), FCID contains additional item-level interdependencies as well. The psychological processes that give rise to FCID and the resultant psychometric properties are discussed. In addition, data from which both normative and ipsative responses were provided by job applicants illustrate very different patterns of correlations as well as very different selection decisions between normative, FCID and ipsatized measures.

References

• Barrick, M. R., Mount, M. K., & Judge, T. A. (2001). Personality and performance at the beginning of the new millennium: what do we know and where do we go next?. International Journal of Selection and Assessment, 9(1‐2), 9-30. PDF
• Bowen, C. C., Martin, B. A., & Hunt, S. T. (2002). A comparison of ipsative and normative approaches for ability to control faking in personality questionnaires. International Journal of Organizational Analysis, 10(3), 240-259.
• Heggestad, E. D., Morrison, M., Reeve, C. L., & McCloy, R. A. (2006). Forced-choice assessments of personality for selection: Evaluating issues of normative assessment and faking resistance. Journal of Applied Psychology, 91(1), 9.
• Meade, A. W. (2004). Psychometric problems and issues involved with creating and using ipsative measures for selection. Journal of Occupational and Organizational Psychology, 77(4), 531-551.

Answer 3

You have seen that one answer is to the question what do they measure (personality traits). The second answer to the question why (because empirical research repeatedly demonstrates that various personality traits can predict various types of job behaviors: mainly performance, deviance, helping, and innovativeness).

But the burning question...how can I fake out these questions to get the job? A: it is unlikely that you can. Some people may tell you to try and guess the correct answer and then to maximize that response (in a forced-choice just guess the right answer). In some old school selection models, this actually makes sense. However, this assumes you know what personality-type the employer is looking for.

The challenge is that modern assessment vendors have a variety of response pattern detectors that are triggered when it looks like someone is trying to guess correctly and max out. These may be actual items that only fakers will pick, like "Everyone likes me" (these are what Christian was talking about), or other more sophisticated detection methods; I work for companies like this, so can't disclose the cooler detection methods.

The hardest thing for the faker is that ultimately you need to know which personality traits are most important and at which level (high,med,low). There is a movement called "evidence-based management" that relies heavily on data and statistical inference in order to justify certain business practices. One such practice is how to weight the personality traits in your assessment battery for employee selection. These weightings are based on statistical models that are proprietary (both the data sources and modeling techniques), may or may not be sample-dependent, and are guaranteed to need revision over the course of a couple of years. This is because employment tests have at best a couple of years of predictive shelf-life. Trying to guess this is like trying to guess the elements of your credit score; well, probably harder.