The Impact of Heavy-tailed Error Distributions on Partially Nested Randomized Controlled Trials Models

Abstract

The Partially Nested Randomized Control Trial (PNRCT) model can be used when the subjects within the treatment group are clustered in groups and the subjects in the control group remained unclustered. The few existing studies have focused on validating, by using Monte Carlo techniques, the efficiency of the PNRCT model compared with other competing models. These studies have been conducted under the assumption that all model assumptions are achieved. One of these assumptions is the normality of error distribution. The literature suggests that real world data hardly ever present a normal distribution. When using data from applied settings, the error distribution from regression models can produce any distributional form. When this happens, the model assumption of normality may be violated, affecting the quality of parameter estimates. This is relevant for the validity of any regression model, because if the error distribution substantially deviates from normality, the model parameter estimates and their standard errors may be seriously affected. Despite the relevance of normality error distribution, very little attention has been given to it in partially nested models. This study assessed the effect of violating the assumption of normality at level 2 in the PNRCT model on the sensitivity of parameter estimates (fixed effects), Type I error rate, and power in the “pure” PNRCT model and in a PNRCT model adjusted by one covariate at level 1. To achieve this goal several conditions such as different number of clusters, different cluster size, and different intra-class correlation (ICC) levels were examined. The results showed robust estimation of the PNRCT model. The fixed effects were unbiased and were more accurately estimated when the number of clusters and subjects increased. The same pattern was found as the ICC increased. The PNRCT model diminishes power up to certain point. This condition is exacerbated in the presence of non-normal distributions. However, as in other studies, power was positively impacted as the number of clusters and number of subjects increased. Finally, the Type I error rate did not substantially deviate from the nominal Type I error rate even for non-normal distributions.