Robustness of ANOVA/F-test
I knew F-test is generally robust against violations of normality. I also knew there are a lot of subtleties in this. But I never looked up the sources, until now. This is a good place to start:
An older classic is:JSTOR: Journal of Educational Statistics, Vol. 17, No. 4, (1992 ), pp. 315-339
Summarizing Monte Carlo Results in Methodological Research: The One- and Two-Factor Fixed Effects ANOVA Cases
Author(s): Michael R. Harwell, Elaine N. Rubinstein, William S. Hayes and Corley C. Olds
Source: Journal of Educational Statistics, Vol. 17, No. 4, Special Issue: Meta-Analysis (Winter, 1992), pp. 315-339
Publisher: American Educational Research Association and American Statistical Association
Stable URL: http://www.jstor.org/stable/1165127
Abstract
Meta-analytic methods were used to integrate the findings of a sample of Monte Carlo studies of the robustness of the F test in the one- and two-factor fixed effects ANOVA models. Monte Carlo results for the Welch (1947) and Kruskal-Wallis (Kruskal Wallis, 1952) tests were also analyzed. The meta-analytic results provided strong support for the robustness of the Type I error rate of the F test when certain assumptions were violated. The F test also showed excellent power properties. However, the Type I error rate of the F test was sensitive to unequal variances, even when sample sizes were equal. The error rate of the Welch test was insensitive to unequal variances when the population distribution was normal, but nonnormal distributions tended to inflate its error rate and to depress its power. Meta-analytic and exact statistical theory results were used to summarize the effects of assumption violations for the tests.
Glass, G. V., Peckman, P. D., & Sanders, J. R. (1972). Consequences of failure to meet assumptions underlying the fixed effects analysis of variance and covariance. Review of Educational Research, 42, 237-288.