WHAT DO SIGNIFICANCE TESTS REALLY TELL US ABOUT THE ENVIRONMENT

Routine application of significance tests does not extract the maximum information from environmental data and can lead to misleading conclu sions. Reasons leading to this are: a significant result can often be reached merely by collecting enough samples; a statistically significa nt result is not necessarily practically significant; and reports of t he presence or absence of statistically significant differences for mu ltiple tests are not comparable unless identical sample sizes are used . These problems are demonstrated by application to pH data for grazed and retired fields, and by discussion of significance tests used in r ecent US regulations for groundwater quality. The advantages of equiva lence tests, where the tester must state the difference of practical d ifference, are discussed and applied to the field pH problem. We recom mend that environmental managers and scientists pay more attention to statistical power and decide on what is a practical difference. Confid ence intervals for the size of the differences, accompanied where nece ssary by equivalence tests, are the preferred means of addressing the question: ''is there a difference of practical significance?''