How can we reduce adverse impacts such as bias, discrimination, and unfairness when using cognitive ability tests and personality tests for selection purposes in an international context?
This is a big and important topic. The following is a bit of an introduction. Feel free to ask a separate more focussed question.
In general distinctions are made between the terms bias, fairness, and discrimination.
- Bias is a statistical term. Bias for a group can indicate that test scores for that group are systematically lower or higher than would be expected from the group's true score. It is often assessed by seeing whether there is a group by test interaction in predicting the criterion. In particular, just because groups differ in means does not mean the test is biased, the groups may differ on the underlying construct.
- Fairness is a moral and ethical construct. There is scope for different people to value different things. For example, using a test that while unbiased , reduces the prevalence of a minority group, may still be seen as unfair. Likewise, affirmative action might be seen as fair in the context of a history of discrimination, yet others may perceive that is being unfair to the non-minority group.
- Discrimination is often used in a legal sense to refer to either direct or indirect forms of discrimination.
Reviewing this area
I strongly recommend you read page 16.16 onwards (from the section labelled "Improved understanding of subgroup differences") of Sackett and Lieven's (2008, PDF) review.
In particular they note that several good predictors of task performance also show subgroup differences. Thus, there is sometimes a trade-off between validity and adverse impact. For a review of subgroup differences see Hough et al, 2001.
I quote a few sections with particular relevance to the question of what can be done to minimise adverse impact:
Sackett et al. (2001) reviewed the cumulative evidence and concluded that several proposed mechanisms are not, in fact, effective in reducing differences. These include (a) using differential item functioning analysis to identify and remove items functioning differently by subgroup, (b) providing coaching programs (these may improve scores for all, but group differences remain), (c) providing more generous time limits (which appears to increase group differences, and (d ) altering test taking motivation.
Sackett et al. (2001) did report some support for expanding the criterion as a means of reducing subgroup differences. The relative weight given to task, citizenship, and counterproductive behavior in forming a composite criterion affects the weight given to cognitive predictors. Sackett et al. cautioned against differential weighting of criteria solely as a means of inﬂuencing predictor subgroup differences; rather, they argued that criterion weights should reﬂect the relative emphasis the organization concludes is appropriate given its business strategy. They also reported some support for expanding the range of predictors used. The strategy of supplementing existing cognitive predictors with additional predictors outside the cognitive domain can reduce the overall subgroup differences in some circumstances. This strategy has received considerable attention because broadening the range of predictors has the potential to both reduce subgroup differences and increase validity
If you're particularly interested in the international context check out Myors et al (2008) which provides an overview of the context of selection and adverse impact from the perspective of a wide range of different countries.
In general, there is also a wide range of general strategies that we might recommend to increase fairness. e.g., Ensure that choice of selection tools is based on a thorough job analysis and an understanding of the competencies required for the job; Use well-validated measures; standardise decision making processes; minimise potential for cheating; etc.
- Hough LM, Oswald FL, Ployhart RE. (2001). Determinants, detection and amelioration of adverse impact in personnel selection procedures: issues, evidence and lessons learned. International Journal Selection & Assessment. 9:152–94.
- Myors, B., Lievens, F., Schollaert, E., Van Hoye, G., Cronshaw, S.F., Mladinic, A., Rodr\'\iguez, V., Aguinis, H., Steiner, D.D., Rolland, F. & others (2008). International perspectives on the legal environment for selection. Industrial and Organizational Psychology, 1, 206-246. PDF
- Sackett PR, Devore CJ. 2001. Counterproductive behaviors at work. See Anderson et al. 2001, pp. 145–64
- Sackett, P.R. & Lievens, F. (2008). Personnel selection. Annu. Rev. Psychol., 59, 419-450. PDF