Trustworthiness of results from statistical analyses relies on the fulfillm
ent of a set of assumptions made about data. A survey of published pedagogy
studies in physical education revealed that examining the assumptions has
been overlooked in data analyses and result reporting. The purpose of this
article is to provide a conceptual understanding of the assumptions and to
summarize available methods to test and address their violations. In the ar
ticle, we present information to show that it is inappropriate to overlook
the assumptions and ignore their violation's impact on interpretation of re
sults. We summarize current findings from theoretical statistical research
that start to challenge the conventional belief about the robustness of tra
ditional inferential statistical analyses. Remedial procedures recommended
in the latest statistical theories are presented for researchers to adopt i
n order to generate valid results from statistical analyses.