A Monte Carlo comparison of stability and accuracy of prediction for six methods of multiple linear regression

Abstract

This study compares five alternative regression methods with OLS. The alternative methods are equal weights, use of the validities as weights (validities method), and one version each of component regression and ridge regression, as well as a new method termed criterion factorization, which is based upon the validities method. Comparisons are made using computer generated data in a fully-crossed five factor experimental design, which included statistical model (fixed and random}, criterion projection (criterion theoretically related to all and just under half the factors underlying the predictors}, sample size (N = 50, 100, and 200), number of predictors (6, 12, and 18), and multicollinearity (high, wide, and low predictor intercorrelations). The relative efficacy of the methods was evaluated in terms of two criteria, namely the expected stability of the regression weights (SEB) and the expected accuracy of prediction in future samples (SPE). The results indicated that differences due to statistical model were negligible, and with very few exceptions, the best performances on the two criteria were provided by ridge regression, criterion factorization, or the validities method. The equal weights method performed poorly relative to the other methods, although its performance with respect to OLS generally followed the findings of previous research. The performance of the principal component regression method examined in the present study was also generally poor, and its use in place of ridge regression, criterion factorization, or the validities method was not recommended. Ridge regression was the method of choice in most of the situations examined in this study. It generally performed poorly when the predictor intercorrelations were low, however, and the validities method is recommended in this situation. With a small sample size and 12 or more predictors with wide or high intercorrelations, criterion factorization was generally the recommended method. The OLS method is recommended whenever p is quite small and sample size is relatively large, particularly when the predictor intercorrelations are not high. Although OLS regression analysis is known to be a powerful data analytic tool, it is concluded that in many situations, allowing a small amount of bias in the estimated regression weights with the use of an alternative regression method can offer substantial improvements through increased stability in the coefficients and improved accuracy of future predictions. Implications and practical applications of the results are discussed.