Caveats and limitations to the indicator and data quality

View the most recent version.

Site selection
Parameter selection
Guideline selection
Sample timing and frequency
Data quality

Site selection

It is recognized that the current collection of monitoring networks was not designed to be representative of Canada and all its watersheds, but to respond to specific federal, provincial, or regional needs. Monitoring sites included in this analysis are almost all located in populated areas and other areas for which it is suspected that water quality is affected by surrounding land uses and other potential stressors, including acid rain deposition, dams, and industries (e.g., pulp and paper and mines). Even so, sites do not comprehensively cover all geographic areas with potential water quality issues or problems across Canada.

From a coverage standpoint, it is unknown what percentage of Canadian lakes and rivers by geographic area or stream flow, is currently represented by the existing 370 monitoring sites. Additionally, each site was weighted equally and independently regardless of location. The only exception to this is the 62 clustered lakes in New Brunswick and Nova Scotia that were aggregated into 7 scores.

Parameter selection

The type and number of parameters included in the WQI calculations differed across the water quality monitoring sites and/or jurisdictions. This flexibility allowed the specific local and regional water quality concerns and objectives of the monitoring programs to be reflected in the WQI scores. However, these differences in parameter selection among jurisdictions/sites make comparability of sites for national aggregation uncertain. It was recommended that between 4 and 15 parameters be measured for the WQI calculation, and this guidance was followed (Environment Canada, 2005b). A recent sensitivity analysis, however, shows that the use of approximately 10 parameters may yield the most stable WQI results (Painter and Waltho, 2005).

In addition, not all possible stressors were sampled everywhere, for several reasons: (1) the random nature of some releases (e.g., unknown or accidental spills); (2) some substances are tracked in other media, such as sediment or fish tissue, that provide more reliable measures; and (3) the high cost of measuring on a routine basis (e.g., for organic substances).

For the Pacific and Yukon Region, metals were removed from the WQI calculation when conditions at a given site were highly turbid. The rationale behind this is the expectation that the high concentrations of metals measured during such events are due to the suspended sediments. These metals are not generally available for biological uptake, and, as such, likely do not pose the same risk to aquatic life as dissolved metals.

Guideline selection

To some extent, exceedances from all parts of Canada for naturally occurring substances (e.g., phosphorous, total suspended solids, and metals) can be due to naturally occurring phenomena, rather than human influence only (Appendix 1 provides a listing of water quality guidelines used in each jurisdiction).

In most cases, metal guidelines are based on measuring total (or extractable) rather than dissolved metals. This conservatively assumes that the full measured amount of the compound is available to be taken up by organisms. However, metals in unfiltered water may be bound to particulates or colloidal molecules and, depending on the chemical species in question, organic materials, making them less bio-available than suggested by a measure of total metals.

Sample timing and frequency

There is variation in timing and frequency of sampling among monitoring programs. Some programs are more intensive to capture the full range of variability/seasonality that is inherent to each site, while others are less intensive, more opportunistic, and/or random, due to resource constraints and the remote nature of some sites. It is not known currently if this poses a problem or creates a bias for the overall indicator. The three-year time period selected as the basis for the indicator accounts for some of this variation and helps to reduce the potential for some sites to “misrepresent” water quality on an annual basis.

A sensitivity analysis conducted on several northern rivers revealed that having fewer samples (i.e., 9) than the required minimum (12) in a three-year period did not produce WQI scores that were significantly different (Glozier et al., pers. comm.).

A sensitivity analysis for southern Ontario streams suggests that more than 12 samples over three years could be required to produce more reliable calculations (Painter and Waltho, 2005).

There were two exceptions made to the minimum 12 samples for the 2002–2004 period. First, three sites in Manitoba were sampled only three times per year due to limited accessibility. Local specialists were confident that the site scores were reliable because of the long monitoring history at these sites. The other exception was for three sites in New Brunswick where only one sample was missed over the three-year reporting period. The other eleven samples were well distributed through the reporting period, and local specialists agreed that the site scores were reliable and the sites should be included.

There were also two sites (one in the Northwest Territories and one Nunavut) that were located at the outflow of large lakes. These sites exhibited behaviour more similar to lakes than flowing waters, i.e., less variability in water quality throughout the year (D. Halliwell, pers. comm.). Thus, the minimum sampling frequency for lakes was adopted for these sites.

Data quality

Water quality data exist at three levels: individual samples taken at monitoring sites; the combination of individual samples to calculate a WQI value for a particular site; and the aggregated data set of all WQI values from the selected sites across the country (see Data management, calculation and verification).

It is inevitable that errors will sometimes occur in individual sample results. The most common are field errors (sample contamination, mislabeling), lab errors (misidentified samples, miscalculations, analytical mistakes) and data entry errors. Each monitoring program follows standardized methods for sample collection in the field to ensure reliability of measurements. Chemical analyses are undertaken in Canadian laboratories accredited by the Canadian Association for Environmental Analytical Laboratories, ensuring analytical methods are up to standard and proper quality assurance/quality control procedures are in place.