Resistance to coaching in forced-choice testing

Robin Orthey, Aldert Vrij, Ewout Meijer, Sharon Leal, Hartmut Blank

Research output: Contribution to journalArticlepeer-review

103 Downloads (Pure)

Abstract

In forced‐choice tests (FCTs), examinees are typically presented with questions with two equally plausible answer alternatives, of which only one is correct. The rationale underlying this test is that guilty examinees tend to avoid relevant crime information, producing a nonrandom response pattern. The validity of FCTs is reduced when examinees are informed about this underlying rationale, with coached guilty examinees refraining from avoiding the correct information but trying to provide a random mix of correct and incorrect answers. To detect such intentional randomization, a “runs” test—looking at the distribution of the number of alternations between correct and incorrect answers—has been suggested but with limited success. We designed a runs test based on distinguishing between patterns that look random and patterns that are random. Specifically, we alternated the horizontal presentation (i.e., presentation left or right on the screen) of the correct answer alternative between each trial. As a consequence, guilty examinees were faced with having to choose to randomize either between correct and incorrect answers—leading to chance performance—or between answers presented on the left or right, producing a pattern that “looks” random. As innocent examinees are unaware of the correct answers, they can only randomize between horizontal positions. Results showed that the number of correct items selected distinguished guilty from innocent examinees only when they were not informed about the underlying rationale. In contrast, alternations between correct and incorrect answers did distinguish informed guilty from innocent examinees. Incremental validity of the alternation criterion and theoretical implications are discussed.
Original languageEnglish
JournalApplied Cognitive Psychology
Early online date13 Aug 2018
DOIs
Publication statusEarly online - 13 Aug 2018

Fingerprint

Dive into the research topics of 'Resistance to coaching in forced-choice testing'. Together they form a unique fingerprint.

Cite this