A critical review of empirical and rational strategies for item selection and keying for biographical data inventories

Abstract

Biographical data (biodata) inventories measure a person's prior experiences under the assumption that these experiences are indicative of knowledge, skills, and abilities relevant for a given purpose, typically selection. A large literature exists on methods for developing biodata inventories and how to weight biodata items, many of which are specific to the context of biodata. The current investigation reviewed the literature on the attributes of biodata inventories and compared empirical and rational methods of scale development, proposing a framework on how to conceptualize the steps involved in developing a biodata inventory. Next, using a number of large archival datasets and simulations, the effectiveness of empirical keying methods hypothesized to improve upon typically-used keying methods was tested. In archival datasets, option-keyed multiple regression was found to explain more variance in cross-validation samples than traditional alternatives, whereas a configural keying method produced results in between the two. The sample size-to-item ratio was an important factor in the extent to which option-keyed regression performed better than alternatives. The results of simulation studies indicated that in many contexts, option-keyed regression produced higher validities than traditional alternatives at 1.5 to 2 times the number of participants as items. As sample size increased, regression explained more than double the variance that traditional methods did. These findings were generally magnified by increases in the number of item response options. Practical implications and recommendations, limitations, and directions for further research are discussed.