Intra-person variability in cognitive test performance

Abstract

Variability in a person's performance on sections or items of a cognitive test is of interest for two reasons. First, previous research suggests that there may be a personal characteristic of performance variability which is correlated with extraversion or impulsivity. Second, the accuracy of a test score may be related to the degree of consistency among scores on items or sections. In the first part of the present research, four speed tests, four accuracy tests, and three personality measures were administered to 402 individuals aged 17 to 40. Between two and eight indicators of intra-person variability (IPV) were calculated for each ability test. IPV scores correlated moderately with test scores, and had low (below .5) split-half reliabilities when test scores were partialed. Correlations between IPV scores on different tests were not significantly greater than zero, regardless of the similarity of the abilities measured by the tests. IPV also failed to correlate significantly with age, sex, or measures of impulsivity, emotional stability or test anxiety. There is no support in these data for the existence of a personal characteristic of variability that manifests itself in cognitive test performance. Analysis of response times to items on one of the unspeeded tests indicates that examinees who had high "continuance" (i.e., were willing to spend a relatively great amount of time on an item) earned higher test scores. However, three aspects of test-taking behavior--continuance, total test time, and the tendency to increase response time as item difficulty increased--were all uncorrelated with IPV scores. The second part of this research investigated the relationship between IPV scores on a vocabulary test and the linear regression of GPA on test scores, for 225 high school students. Contrary to expectation, the regression slope was slightly (but not significantly) higher for the high-IPV cases. Rather than simply indicating the amount of error in the test score, the IPV statistics may indicate systematic bias. Specifically, at low test score levels, high IPV may reflect successful guessing by the examinee, and may therefore indicate that the test score overestimates the examinee's ability.