Estimation of the squared cross-validity coefficient in the context of best subset regression

Abstract

A monte carlo study was conducted to examine the performance of several strategies for estimating the squared cross-validity coefficient of a sample regression equation in the context of best subset regression. Data were simulated for populations and experimental designs likely to be encountered in practice. The results indicated that a formula presented by Stein (1960) could be expected to yield estimates as good as or better than cross-validation, or several other formula estimators, for the populations considered. Further, the results suggest that sample size may play a much greater role in validity estimation in subset selection than is true in situations where selection has not occurred. Index terms: Best subset regression, Cross-validity coefficient, Multiple regression, Predictive validity, Variable selection.