Planned Missingness: A Sheep in Wolf's Clothing

Abstract

There has been an extensive body of methodological literature supporting the effectiveness of planned missingness (PM) designs for reducing survey length. However, in industrial/organizational (I/O) psychology, it is still rarely applied. Instead, when there is a need to reduce survey length, the standard practice is to either reduce the number of constructs measured or to use short forms rather than full measures. The former is obviously unideal. The latter requires prioritizing the measurement of some items over that of others and can also quickly become time and labor intensive, as not all measures have established short forms. This dissertation presents three studies that compare the relatively unused methodology of PM against the common practice of using short forms. First, the two approaches are compared in three archival datasets, finding that PM consistently yields more accurate correlational estimates than short forms. Second, a Monte Carlo simulation is conducted to explore how this comparison may be affected by data characteristics, including the number of constructs, construct intercorrelations, sample size, amount of missingness, as well as different types of short forms. Average of all conditions simulated, short forms produce slightly more accurate estimates than PM when empirically developed short forms are readily available for use. When a part of the sample needs to be used to first develop short forms, the two approaches perform equivalently. When the selection of items for short forms strays from being purely empirical, PM outperforms short forms. Lastly, a qualitative survey exploring social science researchers’ knowledge about PM finds that most are not familiar with PM or have an inaccurate understanding of the concept despite working with surveys frequently. A number of research contexts are identified for which PM may not be suitable. Overall, the findings of this dissertation demonstrate that PM designs are technically effective in producing accurate estimates. Its effectiveness, along with its convenience, makes it a valuable survey design tool. It is apparent that the road to popularizing this technique within the I/O field will require much education in its understanding and application, and this dissertation serves as a first step in doing so.