Effect of variance ratio on ANOVA robustness: Might 1.5 be the limit?

Abstract

Inconsistencies in the research findings on F-test robustness to variance heterogeneity could be related to the lack of a standard criterion to assess robustness or to the dif ferent measures used to quantify heterogeneity. In the present paper we use Monte Carlo simulation to systematically exam ine the Type I error rate of F-test under heterogeneity. One way, balanced, and unbalanced designs with monotonic pat terns of variance were considered. Variance ratio (VR) was used as a measure of heterogeneity (1.5, 1.6, 1.7, 1.8, 2, 3, 5, and 9), the coefficient of sample size variation as a measure of inequality between group sizes (0.16, 0.33, and 0.50), and the correlation between variance and group size as an indica tor of the pairing between them (1, .50, 0, −.50, and −1). Overall, the results suggest that in terms of Type I error a VR above 1.5 may be established as a rule of thumb for con sidering a potential threat to F-test robustness under heteroge neity with unequal sample sizes.