Markov analysis of response timing on a DRL schedule

Abstract

The purpose of the present research was to assess the extent and character of both sequential interresponse time (IRT) dependencies and sequential interreinforcement time (ISrT) dependencies in a DRL-15 sec. schedule of reinforcement. A combined Markov and information analysis was used for this purpose. A major assumption of the analysis was that sequential dependencies could be adequately described by a member of the family of stationary, ergodic, Markov chains. In the analysis, IRTs (or ISrTs) were assigned to a small number of categories, and these categories were considered the "states" of the Markov chain. By computing the maximum likelihood estimates of transition probability matrices up to the second order, it was possible to explore the possibility that certain recognizable sequences of IRTs (or ISrTs) were present in the data. It was also possible to evaluate a number of suggestions made by previous researchers concerning the nature of timing sequences. The information measure of "redundancy" was used to assess the statistical association of these dependencies. Four female albino rats were run on the experimental schedule for 30 sessions. IRTs were automatically key-punched during training and were later analyzed on an IBM 1620 computer. Significant first order sequential IRT dependencies were found for all Ss when the data were dichotomized into reinforcement and non-reinforcement categories, and also when each of these categories was further subdivided into three equal-frequency-bands. The amount of statistical association in both cases, however, was lower than might be desired (generally less than 10%). On a few occasions significant second order dependencies were also found, but again they were accompanied by low statistical association. In general, most of the dependency in the response process was due to a high probability for reinforcements to follow one another, rather than to a tendency for patterns of more molecularly defined IRT categories to occur. The prediction, based on Sidman's (1956) data, that "bursts" would tend to follow long unreinforced IRTs was not substantiated. Some evidence was found, however, that unreinforced IRTs which follow unreinforced IR.Ts are shorter than those which follow reinforced IRTs. Hence, there was some evidence that the first unreinforced IRT which follows a sequence of reinforcements results from S "undershooting" the criterion value. Little evidence of sequential ISrT dependencies was found when a four category Markov analysis was used. ISrT histograms, however, revealed that relative modes develop close to multiples of the criterion value (15 sec.). This finding refutes earlier reports that no such relationship exists in DRL data.