SNEP Low-gradient mIBI Development

A regional collaboration led to the development of a macroinvertebrate IBI for low-gradient streams

Navigation

The ArcGIS StoryMap has multiple sections that can be quickly accessed by using the navigation bar on the top of the page! Highlighted words and phrases are links to external web pages.


Introduction

The Southeast New England region includes the coastal areas in Massachusetts and Rhode Island that are the watersheds of southern Cape Cod, Narragansett Bay, Buzzards Bay, Martha’s Vineyard, and Nantucket. Freshwater streams flow through this region to the coast, providing important habitat for aquatic wildlife, and supporting a variety of uses to humans such as recreation (i.e., swimming, boating, fishing), education, spiritual ceremonies, and drinking water. State agencies, such as the Massachusetts Department of Environmental Protection ( MassDEP ) and the Rhode Island Department of Environmental Management ( RI DEM ), collect water chemistry data and sample biological communities to characterize the condition of streams in this region.

Low-gradient streams in southeastern New England. Photo credit: Tetra Tech.

In southern Massachusetts (MA) and Rhode Island (RI), low-gradient, slow-moving streams that either lack or have infrequent riffle habitat are fairly prevalent. These streams have natural differences in the structure and function of biological communities compared to faster-moving, rocky-bottom streams; therefore, specific bioassessment methods needed to be developed to properly evaluate low-gradient streams. In 2013, MassDEP began sampling the macroinvertebrate community in low-gradient streams after developing a multihabitat collection method. In addition to the sites sampled by MassDEP, the multihabitat collection method was used to sample over 50 sites in low-gradient streams in MA and RI in the summer of 2019. The intent of collecting these data was to compile a dataset that could be used to calibrate a low-gradient IBI for macroinvertebrate assemblages in the Southeastern New England Program ( SNEP ) region.


Purpose and Scope

Under the United States Clean Water Act ( CWA ), state environmental agencies are required to monitor and assess streams and rivers throughout the Nation. Specifically, they are required to assess surface waters for physical, chemical, and biological integrity.

Biological integrity - the capability of supporting and maintaining a balanced, integrated, adaptive community of organisms having species composition, diversity, and functional organization comparable to that of the natural habitat of the region.”

The SNEP Low-gradient IBI Development Project addressed the needs of state agencies to effectively and confidently assess low-gradient, wadeable streams using biological indicators. An index of biotic integrity (IBI) was calibrated using the Reference Condition (RC) approach to recognize characteristics of relatively undisturbed biological samples (Stoddard et al. 2006). This IBI improves the diagnostic ability of state agencies and watershed groups to identify degradation in biological integrity and water quality in southeastern New England low-gradient streams.

Biological and chemical sampling are both used by state agencies to assess stream integrity and identify water quality problems. Photo credit: Tetra Tech.

Indicator Development

Biological community response to increasing stress described in the biological condition gradient (BCG) framework. Photo adapted from USEPA 2016.

To determine when the designated aquatic life use is attained, we use the Reference Condition (RC) approach which compares all streams to the biological condition found in streams with minimal disturbance (Stoddard et al. 2006). We assume that "reference" sites are minimally disturbed and are representative of natural conditions with high biological integrity. Biological integrity can be assessed by sampling multiple biological communities including fish, diatoms, and in this case, macroinvertebrates. We create measurements that summarize aspects of the biological sample that consistently indicate whether a sample is similar to or different from reference conditions. These measurements, or metrics, are combined to form an overall summary – the multimetric index or IBI.

Macroinvertebrates as Indicators

There is ample evidence of effective use of macroinvertebrates as bio-indicators:

  • MassDEP and RI DEM have used macroinvertebrate bio-indicators for many years
  • Neighboring states throughout New England have similar indices
  • Macroinvertebrates are used in waterbody assessments throughout the U.S. and internationally

Environmental impacts can be observed in the composition of the macroinvertebrate community structure and function and can be indicated by:

  • The absence of pollution-sensitive taxa
  • The dominance of a particular taxon - these are often pollution-tolerant taxa
  • Low taxa richness - i.e., the community is not made up of a diverse group of organisms
  • The shift in community composition relative to reference conditions

Macroinvertebrates are useful indicators of integrity and can be reliably used to assess water quality. Photo credit: Tom Danielson, Maine DEP.

Overall Index Development Approach

  1. Find metrics that are consistently responsive to known disturbances
  2. Combine multiple metrics into an index that is accurate in predicting reference and stressed conditions
  3. Use modern, appropriate, and innovative analytical methods to assess index performance
  4. Use the index in the assessment of stream biological integrity
  5. Engage MassDEP and RI DEM experts at every step during development

In this StoryMap, we summarize the development of an IBI for macroinvertebrate assemblages in low-gradient, non-tidal, wadeable streams in the SNEP region. Data collection and index development were done concurrently with the development of a statewide low-gradient IBI for MA. For more detailed information on this project, the low-gradient IBI development report can be found  here . This project was a collaboration between  NEIWPCC  and the Southeastern New England Program ( SNEP ),  Tetra Tech , the Massachusetts Department of Environmental Protection ( MassDEP ), and the Rhode Island Department of Environmental Management ( RIDEM ).

Logos of all collaborating agencies in the low-gradient IBI development project. Photo credit: Tetra Tech.


Results

Rather than bury the lead, we want to present the results of our collaboration before we discuss the considerations and analyses taken to develop this IBI. We do, however, encourage you to continue reading to find out how this IBI was developed. Generally, an IBI consists of metrics used to describe biological assemblages based on common traits or taxonomy. Metrics that are most responsive to anthropogenic stressors are considered and incorporated into an IBI.

A single IBI was developed for low-gradient streams in southeastern New England. The low-gradient IBI can be applied throughout the states of Massachusetts and Rhode Island so long as the IBI applies to site conditions (see Application Potential). The low-gradient IBI comprises six metrics of which three are taxonomy-based and the others are based upon taxa traits such as functional feeding group (FFG), tolerance, and voltanism.

Final metrics selected to be used in the SNEP low-gradient IBI. Table in Jessup et al. 2021.


Data Input

Taxonomic Quality Control

Taxonomy was conducted by two independent, certified taxonomy organizations and overall accuracy was estimated. Report as attachment of Jessup et al. 2021.

Samples were preserved in the field with denatured 95% ethanol and then brought to the lab for sorting and identification.  Cole Ecological, Inc.  processed and identified the samples. As a quality control measure, ten randomly selected samples were independently processed by a second laboratory,  Watershed Assessment Associates . The results met the data quality objectives in the MassDEP and SNEP sampling plans.

Taxonomic Traits and Tolerance Analyses

Macroinvertebrate metrics are often developed based on the frequency of individuals or taxa that are associated with a particular trait (e.g., predators). For trait assignments, we used the attribute table that had been created during the calibration of the MassDEP riffle habitat IBIs as a starting point (Jessup and Stamp 2020). The attribute table included five sets of traits: functional feeding group (FFG), tolerance value, life cycle/voltinism, habit, and thermal preferences. In consultation with RI DEM and MassDEP biologists and contracted taxonomists, we updated some of the phylogeny and taxa names to reflect the most current nomenclature and keys and re-checked the attribute assignments as necessary.

Taxonomic traits used in the SNEP low-gradient project. Table in Jessup et al. 2021.

To inform tolerance value assignments, we ran taxa tolerance analyses on a regional low-gradient dataset that included samples from outside the SNEP region in Massachusetts, Connecticut, Vermont, and New York. Tolerance analyses visualize the shape of the taxon-stressor relationship across a continuous numerical scale and can be used to identify tolerance optima and limits (Yuan 2006). MassDEP and RI DEM biologists reviewed the results and assigned taxa to three tolerance categories: intolerant, intermediate, and highly tolerant.

An example of taxa tolerance analysis output that visualizes the shape of the stressor-response analysis for each taxon. Photo credit: Ben Block, Tetra Tech.


Index Calculation

Index development required multiple steps:

  1. Define the disturbance gradient
  2. Site classification
  3. Metric calculation and selection
  4. Metric responses and scoring
  5. Index testing and performance
  6. Sub-sample size assessment
  7. Condition threshold development

Site Classification

Site classification addresses the recognition that there might be different expectations of the sampled benthic assemblage due to natural effects and influences. Natural variation in stream slope, size, dominant substrates, and other factors might cause a sample to contain more or less of certain taxa groups. Metrics derived from taxa traits would likely exhibit variation due to these natural, ecoregional differences in stream characteristics. When we use macroinvertebrates to indicate biological conditions relative to disturbance, we attempt to account for different expectations due to the background natural setting. Comparisons of metrics between reference sites and those with high disturbance will be more sensitive to stressors if the natural variation is filtered out through site classification.

There was no strong indication of natural variation present in the low-gradient dataset, therefore, no site classes were made and the resulting IBI is applicable throughout the SNEP region.

Example of a low-gradient stream in southeastern New England. Photo credit: MassDEP.

Metric Calculation and Selection

During the calibration of the SNEP low-gradient IBI, a parallel project (statewide MassDEP low-gradient IBI) was also underway. Several members of the SNEP workgroup were also members of the MassDEP workgroup. There was also overlap across the two datasets (the SNEP samples were included in the statewide MassDEP IBI dataset). Thus, the two projects were not completely independent and often were informing one another.

Metrics were calculated in the BioMonTools R package, developed by Erik Leppo (Tetra Tech). Using BioMonTools, Tetra Tech calculated well over 150 macroinvertebrate metrics for the IBI, but more metrics were calculated and evaluated than ended up in a final index.

Metrics were grouped into six categories:

  • Richness
  • Composition
  • Tolerance
  • Functional feeding groups
  • Habit
  • Life Cycle

Metrics were evaluated and selected based on the following:

A visual example of discrimination efficiency (DE). Photo credit: Tetra Tech

  • Sensitivity - How well did the metric distinguish between reference and stressed sites (evaluated using DE and z-score response metrics)? What was the relationship between metric and disturbance variables (consider the direction and strength/significance of response)?
  • Redundancy - Evaluated using Spearman Rank Correlation
  • Representation across metric categories
  • Precision - Evaluated using  Coefficient of Variation (CV) 
  • Metrics being considered in the concurrent MassDEP low-gradient IBI Project

Sensitivity measures: Discrimination Efficiency (DE) is calculated as the percentage of metric scores in stressed sites that are worse than the worst quartile of those in the reference sites. The Z-score is calculated as the difference between mean reference and stressed metric or index values divided by the standard deviation of reference values.

Metric Responses and Scoring

Formulae were applied to the metrics to standardize them to a 100-point scoring scale. The scoring scale was based on the percentile statistics (and minimum values) of metric values across all sites (as opposed to only reference sites).

For metrics that decreased with increasing stress (referred to as ‘decreasers’; an example is the number of intolerant taxa metric), we used the following equation in which the 95th percentile was the upper end of the scoring scale and the minimum possible value (zero) was the lower end.

Scoring formulae for decreasing metrics. Photo credit: Tetra Tech.

For metrics that increased with increasing stress (referred to as ‘increasers’; an example is the number of tolerant taxa metric), we used the following equation in which the 95th percentile was the upper end of the scoring scale and the 5th percentile was the lower end

Scoring formulae for increasing metrics. Photo credit: Tetra Tech.

Index Testing and Performance

Indices comprise multiple metrics to assess different aspects of the biological community. Index formulations were created and evaluated in two ways:

  • Manual metric substitutions - where metrics were added/removed manually and assessed for performance
  • Automatic all-subsets routine - an R script that combines all potential combinations of metrics into indices and evaluates performance on each

An example of the all-subsets routine in R. Photo credit: Tetra Tech.

Index alternatives were screened for discrimination performance, the diversity of metric categories, and redundancy. The ideal index performs well and measures multiple aspects of the biological assemblages. Two state biologists culled the list of over 100,000 indices to approximately 20 index options that were discussed within the working group.

Index verification by plotting IBI scores by % forest cover. Figure in Jessup et al. 2021.

Often, index performance is assessed by applying the index to validation samples. In this project, however, there were not enough samples to allow for a validation dataset, so index performance was assessed only using correlation. The IBI was strongly correlated to land-use stressors such as conductivity and percent urban and agricultural land cover, measures of watershed integrity (i.e., IWI/ICI), and habitat integrity (i.e., habitat scores and percent forested land cover). We found the IBI to respond as expected to natural and stressor variables.

Sub-sample Size Assessment

The low-gradient IBI was developed to be responsive to different subsample sizes. After an index was selected by the working group, Tetra Tech performed an additional analysis on the full dataset to evaluate whether the IBI was affected by subsample size – some regional partners standardize samples to different subsample counts. Of particular interest was the effect on the two richness metrics originally included in the index, because the number of taxa found in samples generally decreases with a decrease in the number of individuals collected (Gotelli and Graves 1996).

The working group decided to replace the richness metrics with percent taxa metrics. Although Model 6_13784 (NumTaxaIBI – containing richness metrics) was initially selected by the working group through the all-subsets model routine, the PctTaxaIBI alternative (containing percent taxa metrics) was decided upon as the final model. This index eliminated the need to adjust metric scoring formulae due to subsample size and simplified the application of the IBI across the region (Block et al. 2020). This index was also chosen for the MassDEP low-gradient IBI.

Richness metrics were replaced by relative richness metrics due to concerns of subsample size. Table in Jessup et al. 2021.

Condition Threshold Development

Once site classes are established and indices are calibrated, some entities establish thresholds for numeric biocriteria. We used multiple analyses to identify possible thresholds associating ranges of index values with biological condition categories. Multiple lines of evidence were used to estimate condition categories including reference distribution statistics, balancing Type I and Type II error rates, and proportional odds logistic regression.

Numeric condition thresholds for the low-gradient IBI. Table in Jessup et al. 2021.

Using the reference condition approach, a candidate threshold value was identified that broadly separated stressed sites from reference sites. In addition, secondary thresholds were identified within the generally unimpacted and generally impacted index ranges which allows for refined emphasis on biological condition when prioritizing or justifying management decisions. Below are the proposed condition thresholds for the low-gradient IBI.

Visual representation of condition thresholds. Figure in Jessup et al. 2021.


Application Potential

IBIs are numeric representations of biological conditions used to indicate biological impairment and low scores are directly related to increased human disturbance. Index narratives derived from condition thresholds can be used as assessment designations. Exceptional and satisfactory communities can be assessed as “supportive”, whereas moderately degraded and severely degraded communities as “impaired.” The low-gradient IBI can be applied throughout the states of Massachusetts and Rhode Island so long as the IBI applies to site conditions (see below).

Criteria for low-gradient streams:

  • Geographic area: Massachusetts and Rhode Island
  • Stream type: perennial, freshwater, wadeable, low-gradient, slow-moving streams with soft or hard substrate, with at least one of the following habitats: snags, root wads, leaf packs, aquatic macrophytes, undercut banks, overhanging vegetation, or hard bottom.
  • Subsample size: 300-count samples are recommended for best performance, but the IBI can also be applied to 200 or 100-count samples
  • Taxonomic resolution: lowest practical level
  • Collection gear: Aquatic Kick Net with 500‑μm mesh
  • Collection method: 10 kicks, sweeps, and/or jabs from multiple habitats (listed above) taken over a 100-m reach and then composited into a single sample. Habitats are sampled in proportion to their occurrence
  • Collection period: July 1–September 30

Example of low-gradient stream in southeastern New England. Photo credit: Tetra Tech.

Application Tools

Tetra Tech developed a bespoke R Shiny app that allows RI DEM, MassDEP, and other SNEP partners to efficiently calculate IBI scores for any new data they possess. Shiny apps are interactive web applications that are linked to R software, which is an open-source programming language and software environment for statistical computing. The app outputs metric values and scores, index scores, and contains mapping and reporting capabilities. Also, only an internet connection is required – no software or downloads are necessary. Check it out via the link below!


Additional Resources

Here are additional resources available to explore the development of the SNEP low-gradient IBI. Below is a link to a non-technical StoryMap that describes the context of IBI development in the broader context of biomonitoring. For additional detail and reading, you can download the full report (and data) from GitHub.


Acknowledgments and Credits

The Low-Gradient Coastal Index of Biotic Integrity (IBI) for Wadeable Waters in Southern New England project was supported by the Southeast New England Program (SNEP) Watershed Grants. SNEP Watershed Grants are funded by the U.S. Environmental Protection Agency (EPA) through a collaboration with Restore America's Estuaries (RAE) and awarded to the New England Interstate Water Pollution Control Commission [now NEIWPCC]. For more on SNEP Watershed Grants, see www.snepgrants.org.

This ArcGIS StoryMap and the low-gradient IBI development project were funded by a grant from the EPA and SNEP. The contract to Tetra Tech was facilitated by Maryann Dugan of NEIWPCC. The index development team consisted of biologists from the Massachusetts Department of Environmental Protection (MassDEP) (James Meek, Allyson Yarra, Robert Nuzzo) and the Rhode Island Department of Environmental Management (RIDEM) (Katie DeGoosh, Jane Sawyers). Michael Cole of Cole Ecological, Inc. provided feedback on taxa attribute assignments and nomenclature questions and worked with Kelly Nolan and Natalie Ohl (Watershed Assessment Associates) on the taxonomic identification quality control process. In addition, several state partners from the Northeast provided low-gradient data that were used in the regional taxa tolerance analyses. Regional partners included Ansel Aarrestad and Chris Bellucci from the Connecticut Department of Energy & Environmental Protection, Aaron Moore from the Vermont Agency of Agriculture Food and Markets (AAFM), Steve Fiske (retired, formerly of the Vermont Department of Environmental Conservation), and Gavin Lemley, Brian Duffy, and Zachary Smith from New York Department of Environmental Conservation. We are very grateful for the hard work and enthusiasm of all the project participants. 

Photos

Tetra Tech; or Tom Danielson, Maine DEP

Block et al. 2020

Block, B. D., J. Stamp, and B. K. Jessup. 2020. MassDEP/SNEP index selection and evaluation with consideration of sub-sample sizes. Prepared for Massachusetts Department of Environmental Protection and Southeast New England Coastal Watershed Restoration Program.

Gotelli and Graves 1996

Gotelli, N. J. and G. R. Graves. 1996. Null models in ecology. Washington, DC: Smithsonian Institution Press.

Jessup et al. 2021

Jessup, B., B. D. Block, and J, Stamp. 2021. Development of an Index of Biotic Integrity for Macroinvertebrates in Freshwater Low Gradient Wadeable Streams in Southern New England Prepared for NEIWPCC, Worcester, MA. Prepared by Tetra Tech, Montpelier, VT.

Jessup and Stamp 2020

Jessup, B. and J. Stamp. 2020. Development of Indices of Biotic Integrity for Assessing Macroinvertebrate Assemblages in Massachusetts Freshwater Wadeable Streams. Prepared for the Massachusetts Department of Environmental Protection, Worcester, MA. Prepared by Tetra Tech, Montpelier, VT.

MassDEP 2004

MassDEP. 2004. CN 187.1. QAPP for 2004 Biological Monitoring and Habitat Assessment.

Nuzzo 2003

Nuzzo, R. 2003. CN 32.2. Standard Operating Procedures: Water Quality Monitoring in Streams Using Aquatic Macroinvertebrates. Massachusetts Department of Environmental Protection, Division of Watershed Management. Worcester, MA. 35p.

Stoddard et al. 2006

Stoddard, J. L., D. P. Larsen, C. P. Hawkins, R. K. Johnson, and R. H. Norris. 2006. Setting expectations for the ecological condition of running waters: the concept of reference condition. Ecological Applications 16:1267–1276. 

Tetra Tech 2019

Tetra Tech. 2019. Sampling and Analysis Plan - Data Collection for Development of an Index of Biotic Integrity for Freshwater Low-Gradient Wadeable Streams in Southern New England. Prepared for the New England Interstate Water Pollution Control Commission, Lowell, MA. Prepared by Tetra Tech, Montpelier, VT.

USEPA 2016

USEPA. 2016. A Practitioner’s Guide to the Biological Condition Gradient: A Framework to Describe Incremental Change in Aquatic Ecosystems. EPA-842-R-16-001. U.S. Environmental Protection Agency, Washington, DC

Yuan 2006

Yuan, L. 2006. Estimation and Application of Macroinvertebrate Tolerance Values. Report No. EPA/600/P-04/116F. National Center for Environmental Assessment, Office of Research and Development, U.S. Environmental Protection Agency, Washington, D.C.

Low-gradient streams in southeastern New England. Photo credit: Tetra Tech.

Biological and chemical sampling are both used by state agencies to assess stream integrity and identify water quality problems. Photo credit: Tetra Tech.

Biological community response to increasing stress described in the biological condition gradient (BCG) framework. Photo adapted from USEPA 2016.

Macroinvertebrates are useful indicators of integrity and can be reliably used to assess water quality. Photo credit: Tom Danielson, Maine DEP.

Logos of all collaborating agencies in the low-gradient IBI development project. Photo credit: Tetra Tech.

Final metrics selected to be used in the SNEP low-gradient IBI. Table in Jessup et al. 2021.

Taxonomy was conducted by two independent, certified taxonomy organizations and overall accuracy was estimated. Report as attachment of Jessup et al. 2021.

Taxonomic traits used in the SNEP low-gradient project. Table in Jessup et al. 2021.

An example of taxa tolerance analysis output that visualizes the shape of the stressor-response analysis for each taxon. Photo credit: Ben Block, Tetra Tech.

Example of a low-gradient stream in southeastern New England. Photo credit: MassDEP.

A visual example of discrimination efficiency (DE). Photo credit: Tetra Tech

Scoring formulae for decreasing metrics. Photo credit: Tetra Tech.

Scoring formulae for increasing metrics. Photo credit: Tetra Tech.

An example of the all-subsets routine in R. Photo credit: Tetra Tech.

Index verification by plotting IBI scores by % forest cover. Figure in Jessup et al. 2021.

Richness metrics were replaced by relative richness metrics due to concerns of subsample size. Table in Jessup et al. 2021.

Numeric condition thresholds for the low-gradient IBI. Table in Jessup et al. 2021.

Visual representation of condition thresholds. Figure in Jessup et al. 2021.

Example of low-gradient stream in southeastern New England. Photo credit: Tetra Tech.