One of the first steps in stats is appreciating the gestalt of these data points to determine their distribution, or frequency of observed values. Many popular stats tests are based on the assumption that data follow a “standard normal distribution”. If data do not meet this assumption, then results of commonly applied stats tests might be less meaningful.

.

One-way ANOVAs measure the ratio of variance across groups to variance within each group, helping us answer: Do >2 groups differ on a factor? Say we're interested in whether happiness differs among 3 groups after each listened to a 2010s viral song repeatedly for 2 hrs. Group A listened to “Gangnam Style”, Group B listened to "Baby Shark", and Group C listened to "What Does the Fox Say?" Using a one-way ANOVA, we test: does happiness after listening to a viral song differ based on song choice?

The “C” in ANCOVA signals that we are “covarying” for other factors that might influence an outcome separately from experimental factors. In our study, we might control for depression symptoms – a person with more severe depression might have blunted happiness ratings not related to song choice. Using an ANCOVA, we test: does happiness after listening to a viral song differ based on song choice when accounting for depression severity?

Repeated measures refers to data collected over 2+ time points for the same factor. Our example’s repeated measure would be happiness ratings before AND after the listening period. rmANOVA tests for changes in a factor within one group or among 2+ groups. We can test main effects (i.e., do groups differ in happiness when combining time points; does happiness change across time points when combining all groups) and, importantly, test for an interaction effect (i.e., do groups differ in how they change on a factor over time). The image above shows an interaction effect. Groups A and B do not differ on happiness before music. Group A’s happiness remains stable after 2hrs of Gangnam Style, but Group B’s happiness significantly decreases after 2hrs of Baby 🦈.

[4] Yale University Statistical Topics “The Normal Distribution”

[6] Lew, M. J. (2013). To P or not to P: on the evidential nature of P-values and their place in scientific inference. arXiv preprint arXiv:1311.0081.

[7] Christenson, P. (1995). To p or Not to p. Journal of Child and Adolescent Psychiatric Nursing, 8(1), 42-42.

[8] Wasserstein RL, Lazar NA. The ASA’s statement on p-values: context, process, and purpose. Am Stat. 2016;70:129-133.

[9] Lin, M., Lucas Jr, H. C., & Shmueli, G. (2013). Research commentary—too big to fail: large samples and the p-value problem. Information Systems Research, 24(4), 906-917.

[10] Greenland, S., Senn, S. J., Rothman, K. J., Carlin, J. B., Poole, C., Goodman, S. N., & Altman, D. G. (2016). Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations. European journal of epidemiology, 31(4), 337-350.

[12] New England Complex Systems Institute, “Concepts: Linear and Nonlinear”

[13] Swinscow TDV. In: Statistics at square one. Nineth Edition. Campbell M J, editor. University of Southampton; Copyright BMJ Publishing Group 1997.