Predicting programming performance in novice programmers by measures of cognitive abilities

Abstract

This research explores the relationships between profiles of individual differences in cognitive functioning (cognitive structures, cognitive capabilities and information-processing characteristics) and measures of programming skill in order to understand how novice FORTRAN programmers implement these cognitive capabilities in the performance of programming tasks. Cognitive abilities were assessed via seven standardized tests of cognitive factors measuring (1) flexibility of closure, (2) induction, (3) associative memory, (4) syllogistic reasoning, (5) spatial scanning, (6) speed of closure and (7) general verbal aptitude. These factors were included on the basis of previous research indicating their potential explanatory power for either programming or programming-like tasks. Programming ability was measured by a test designed by the author and intended to measure ability on a range of programming subtasks originally suggested by Schneiderman and Mayer (1979). This criterion instrument was subjected to principal components analysis to uncover eighteen first-level subtask groupings. Subsequent higher-order analyses resulted in nine and four subtask components, respectively. Within each set of components, the subtasks represented were of an increasingly conceptual nature, and, of course, of a decreasingly mechanistic nature. Additionally, a total score and scores for other models of subtask categorizations were computed. Regression analysis was performed to predict criterion scores from scores on the predictor (cognitive ability) battery. Results of these analyses indicated that ability on the syllogistic reasoning dimension is of particular importance in tasks requiring comprehension of the logic of an existing computer program (or fragment) and in the ability to compose a program, i.e., in the translation of logic to code, or vice-versa. Ability on the dimension of induction is also important here. It was also shown that debugging skills are to some degree dependent upon ability on the dimension of spatial scanning. Additionally, general verbal aptitude is seen as an important predictor of overall programming performance. These results are described in terms of hypotheses from the general literature concerning task/ability relationships and in regard to their implications for both immediate application and future research.