Supervised Online Testing – A Way To Reduce Faking?

An international between-subject study by Matthias Stadler & Psytech International

Introduction

Online testing is used more and more in personnel selection. However, research has only just begun to explore the possible gains and risks from taking psychometric tests out of standardized settings. Tippins et al. (2006) mention several unresolved issues regarding unproctored internet testing, especially in high-stakes situations. Beaty et al. (2011) criticized that especially the possible effect of the higher anonymity of unproctored online testing on the validity of psychometric testing have not been investigated much so far.

To augment this sparse body of knowledge, we report data on both Chinese and European applicants and investigate the effect of increased supervision on the personality scores of high-stakes applicants. It is assumed, that a higher degree of supervision will lead to a less stereotypical score on both the most relevant personality traits and the fake-good scales (Mahar et al., 2006) but will not effect the social desirability scores.

Methods

Participants

• four large independent samples:
  • European – Unsupervised (N = 13,929)
  • European – Supervised (N = 1,009)
  • Chinese – Unsupervised (N = 14,003)
  • Chinese – Supervised (N = 1,826)
• Real applicants to a multitude of different managerial positions
• European samples aged between 18 and 92 (M = 37.96) 29.6% female; Chinese samples aged between 18 and 85 (M = 29.77), 34.8% female.

Questionnaire

• All participants completed the 15FQ+
• Big Five personality traits were obtained by factor analysis of the 16 scales
• The Fake Good scale was derived from specific item responses within the test.

Supervision

• Unsupervised condition: The test administrator issues an invitation by email to a respondent at a remote location. The questionnaire is completed at the convenience of the respondent and the results are made available online to the administrator.
• Supervised condition: Individual on screen testing with the administrator present.

Results

Fig. 1 Effect of supervision on neuroticism and conscientiousness

Fig. 2 Effect of supervision on social desirability and Fake Good

Discussion

Main findings
Both the European and the Chinese samples showed very similar patterns. In both cases the participants in the unsupervised condition displayed significantly higher conscientiousness (Chinese: d=.16; European d=.18) and lower neuroticism scores (Chinese: d=.20; European d=.19).
According to Peterson et al. (2011) these two traits are the most vulnerable to faking, while social desirability scales should not show any differences. Supporting this there were no significant differences in the social desirability scales; however the participants in the unsupervised condition scored significantly higher on the Fake Good scale (Chinese: d=.15; European d=.16).
The significant differences on the Fake Good scale supports the hypothesis, that higher supervision reduces self presentation (faking), while the social desirability scales are not able to detect it.
These findings are consistent with Mahar et al. (2006) who found that faking would lead to a more stereotypical pattern of personality and to elevated Fake Good scores.

Limitation
Causal interpretation of the results is limited due to the quasi-experimental design. However there were no signs of systematic differences between the two conditions and the very large sample sizes should eliminate all random errors.

References
Beaty, J. C., Nye, C. D., Borneman, M. J., Kantrowitz, T. M., Drasgow, F., & Grauer, E. (2011). Proctored Versus Unproctored Internet Tests: Are unproctored noncognitive tests as predictive of job performance?. International Journal Of Selection & Assessment, 19(1), 1-10. doi:10.1111/j.1468-2389.2011.00529.x
Mahar, D., Coburn, B., Griffin, N., Hemeter, F., Potappel, C., Turton, M., & Mulgrew, K. (2006). Stereotyping as a response strategy when faking personality questionnaires. Personality and Individual Differences, 40, 1375–1386.
Peterson, M. H., Griffith, R. L., Isaacson, J. A., O’Connell, M. S., & Mangos, P. M. (2011). Applicant faking, social desirability, and the prediction of counterproductive work behaviors. Human Performance, 24, 270–290.
Tippins, N. T., Beaty, J., Drasgow, F., Gibson, W. M., Pearlman, K., Segall, D. O., & Shepherd, W. J. (2006). Unproctored Internet testing in employment settings. Personnel Psychology, 59, 189–225.