Using the normal range as a criterion for ecological significance in environmental monitoring and assessment

Data from monitoring programs are often used to compare a potentially impac ted location with unimpacted: reference locations. Regardless of the experi mental design, both statistical and biological criteria are frequently used to judge the significance of observed differences. In this paper, we defin e ecologically relevant differences as observations from impact locations t hat fall outside the normal range of variation based on reference-location data. We also define the normal range as the region enclosing 95% of the po pulation of reference-location observations. This 95% region can then be ex pressed generically as either standard deviations (univariate) or generaliz ed distances (multivariate). Such re-expression allows far the construction of appropriate null hypotheses and statistical tests that determine the pr obability that a test location Falls within the normal range. We evaluate t he ability of three statistical tests (traditional two-sample contrast, and non-traditional equivalence and interval tests) to detect when test locati ons lie outside of the normal range of variation. For locations that are tr uly outside of the normal range, traditional two-sample contrasts will lead to erroneous conclusions of no impact about 50% of the time when there are 10-20 reference locations in the design. In contrast, for locations truly outside the normal range of variation, equivalence tests will lead to erron eous conclusions of no impact at most 5% of the time, regardless of sample size. The penalty associated with this test is that locations that are trul y just inside the limits of the normal range will have a high probability o f being declared impacted. For locations truly inside the normal range of v ariation, interval tests will lead to a conclusion of impact at most 5% of the time. The penalty associated with this test is chat locations that are truly impacted may not be declared impacted unless effects are very large. When evaluating an impacted location with respect to a set of reference loc ations, we caution that one must consider these characteristics of the test s when attempting to judge significance.