Montana Heat Vulnerability Index

As a portfolio project I created a heat vulnerability index (HVI) for the state of Montana at the Census Tract level, using a methodology that is as intuitive and practically useful as possible.

Given Montana's warming climate, it's important to identify the locations of communities and populations that experience increased health vulnerability to extreme and prolonged heat within Montana. This index can be useful for implementing potential interventions and as a guide for community stakeholders and emergency planning.

My methodology closely follows the approach developed in 2023 by the  New Jersey Climate Change Resource Center  for their statewide New Jersey heat vulnerability index. Use these links to view their  interactive map , detailed  methodology , and  project overview video . That methodology document, especially, is a rich source of information on the development of vulnerability indices such as these and the existing literature on heat-specific vulnerability that informed the NJ (and subsequently my) approach.

What I found particularly compelling and useful is how they developed an index that can be broken down into its three component categories that together constitute vulnerability to heat. These categories are Exposure, Sensitivity, and Adaptive Capacity.

Vulnerability indices can be powerful tools for guiding discussion, promoting community engagement in resilience planning, and even prioritizing areas for specific interventions. However, they're highly subjective tools. Their usefulness depends on the assumptions, data, and methods that form their foundations. If any of these parameters aren't appropriate for a given place and context, a vulnerability index may at best be less useful than it could be, and at worst directly harmful as it could provide a misguided sense of where vulnerability truly lies.

To illustrate this idea, in the  Comparison  section I analyze the main method that follows the New Jersey methodology, what I'll call the NJ Equal-Weight approach, with two other approaches, an Unweighted TOPSIS multi-criteria decision analysis that provides an estimate of uncertainty to the NJ approach, and also a Weighted TOPSIS method that follows the vulnerability assessment methodology in the 2021 Montana Climate Assessment which places more weight on current and future exposure to extreme heat.

Exposure represents the physical environmental stressors or characteristics that lead to worse health outcomes at the individual and community level. An example is the average summertime temperature for an area.

Sensitivity is the degree to which individual or communities may be affected by extreme heat. Examples include how elderly people and those with certain chronic medical conditions are more susceptible to harmful health outcomes due to heat.

Adaptive capacity is the ability of the individual or community to respond to and take action to mitigate the hazards associated with extreme heat and recover from an extreme heat event. Examples that decrease this include poverty and lower education levels, while examples that increase adaptive capacity include stable employment and having health insurance.

Creating a composite HVI in this manner provides valuable, actionable insight for community members and practitioners to understand vulnerability not only at a high level, but also how these specific categories are driving that vulnerability in different communities throughout the state.

The complete list of categories and indicator variables I used follow in the tables below.

Data from the CDC PLACES program for crude prevalence of diabetes, asthma, and coronary heart disease is from 2021, but was mapped to the pre-2020 Census Tract boundaries. For the 2020 decennial census, the Census Bureau expanded the number of tracts in Montana from 271 to the present-day total of 319. All other data sources besides CDC Places were sourced with the correct mapping to the post-2020 tract boundaries.

To overcome this discrepancy with the PLACES data while sticking to a scope of using publicly-available data, I manually reviewed these changes in tract boundaries across the state using GIS tools. In all cases except one, the additional tracts came from splitting a pre-2020 tract into 2-3 smaller subsections, presumably to account for population growth. In all these cases, I assigned the pre-2020 data values for diabetes, asthma, and coronary heart disease to each of the new, smaller tracts that now reside within the older pre-2020 tract. For instance:

There was only one case where multiple pre-2020 tracts were condensed rather than split. In this case I took the respective average diabetes, asthma, and coronary heart disease values from the two pre-2020 tracts and averaged them to create the values for the single post-2020 tract they formed.

Other than this wrinkle I followed the same methodology as the NJ team regarding transforming all indicator variables to approximate a normal distribution (which limits the effect of outlier data points and skewed distributions), then standardizing each indicator and averaging them within each of the three categories to create group composite indicator scores. These group scores were again z-score transformed and mean centered to ensure that each group score was in the same units (standard deviations above and below the mean) and averaged into the final composite HVI — as shown in this process diagram from the NJ team.

Ultimately, this is only a single method for combining the impacts of these indicator variables. Following the New Jersey approach, I combined the indicators using the assumption that they all contributed equally (aka that they’re “weighted” equally) to vulnerability. This is a standard approach when there is inconclusive or insufficient research to support weighing certain indicators more than others, yet this obviously may not exactly align with reality. As the saying goes — all models are wrong, but some are useful. One way to check the validity of this equal-weight approach is through Monte Carlo simulations, where you use thousands of simulations to test the effect that weighing the indicator variables differently has on the resulting vulnerability scores for each Census tract. In each simulation the indicator variables are given a certain weight and then combined to form an overall estimate of vulnerability. The resulting pattern of vulnerability scores for each tract from all of these simulations provides a key piece of evidence — it shows how susceptible the vulnerability score is to different combinations of weighted indicators. A relatively small spread of possible vulnerability scores indicates that we can have higher confidence that the equal-weighting approach gives a plausible portrayal of vulnerability. Without more research to draw on we can’t say what the true weights of all the indicators variables should be, but we can say how much different combinations of weights matter. Put in the form of a question, this answers “how much does vulnerability to extreme heat depend on how we emphasize or de-emphasize the different indicator variables?”

I put together a series of short intro videos describing the aspects of the sensitivity analysis.

To test this I ran 10,000 Monte Carlo simulations of heat vulnerability using the Technique for Order of Preference by Similarity to Ideal Solutions (TOPSIS) method (Sevachandran et al. 2018). I used this method to test two different scenarios: an equal-weight TOPSIS analysis using the same set of indicator variables as the NJ Equal Weight approach, and secondly a weighted TOPSIS analysis following the 2021 Montana Climate Assessment approach.

In the same manner as the NJ Equal Weight approach, in both scenarios the resulting vulnerability scores from the algorithm were grouped into five categories according to the percentile of the data that each score resided in.

Vulnerability Categories Low: 0-20th percentile Moderate Low: 20-40th Moderate: 40-60th Moderate High: 60-80th High: 80-100th

Here I used the same set of indicator variables as the NJ Equal Weight method and all indicators were initially weighted equally. I ran the TOPSIS algorithm for 10,000 iterations, where each of the indicator variables were randomly assigned a weight using a Dirichlet distribution (a family of continuous multivariate probability distributions representing the indicator variables) and then forced the sum of the weights equal to 1. The algorithm outputs a "closeness" score between 0 and 1 for each simulation, with higher values indicating more vulnerability. The tracts were assigned a vulnerability category based on which percentile of the dataset their median closeness score fell within.

This scenario is based on the  2021 Montana Climate Assessment "Climate Change and Human Health in Montana" report.  See Appendix A for the detailed explanation of their Multi-Criteria Decision Analysis of vulnerability at the county level. In this scenario, the TOPSIS method is modified to place more emphasis on exposure to extreme heat, and it adds an additional indicator variable that represents future changes in heat projections. This variable is the heat index values by UCS (2019) for the 100°F threshold at mid century with the RCP8.5 scenario (i.e. no action is taken to reduce emissions). The heat indices for each county were generated based on an ensemble of general circulation models and include a combination of temperature and humidity estimates to create a “feels like” temperature rating commonly used when estimating heat impacts on humans.

My approach here differs from the 2021 MCA analysis in three ways. I analyze at the census tract level rather than the county level, so all county-level data (including the UCS predictions of future extreme heat) are downscaled from the county to their respective tracts. I also keep the full suite of variables from the NJ Equal Weight approach rather than the smaller set of variables that the 2021 MCA used.

Importantly, this scenario places more emphasis on current and future projected exposure to extreme heat. Both the average summer land surface temperature and the future extreme heat days are given fixed weights of 0.25 apiece to ensure that exposure to current and future heat makes up roughly half of the impact on the overall vulnerability score. The remaining variables from the exposure, sensitivity, and adaptive capacity groups cumulatively make up the remaining weight of 0.50.

The tracts were assigned a vulnerability category in the same manner as the Unweighted approach, with the vulnerability category based on which percentile of the dataset the tract's median closeness score fell within.

The following charts display key indicators for how well the NJ Equal Weight approach and the Unweighted and Weighted TOPSIS approaches agree on how heat vulnerability should be assigned for each tract across the state.

I find three key takeaways in this analysis. The first is that the Unweighted TOPSIS approach provides evidence supporting the robustness of the main NJ Equal Weight approach I employ for mapping vulnerability across the state. Across 10,000 Monte Carlo simulations of differently-weighted indicator variables, the Unweighted TOPSIS algorithm generally estimates vulnerability in the same way as the NJ Equal Weight approach, which helps us have confidence that the assumption of equally-weighted indicator variables that the NJ approach is built upon is indeed valid. In general, the uncertainty results suggest that when clumped into the five categories the results are robust and that most of the categories stay the same irrespective of the weighting scheme.

Secondly, the Weighted TOPSIS approach that follows the 2021 MCA also provides evidence in support of the NJ approach. While there is more disagreement between this method and the NJ Equal Weight approach, there is still broad agreement.

Lastly, comparing the three approaches demonstrates how much the assumptions underlying estimations of vulnerability matter. While together these three approaches generally agree on how vulnerability should be assigned, clear differences emerge in the Weighted TOPSIS approach versus the other two for tracts in the hottest and fastest-warming areas of the state.

Whether a heat vulnerability index should be built to identify current vulnerability, as the NJ Equal Weight method does, or also have an future-focused element that takes into account indicators of future vulnerability is an open question that can only be resolved by working hand-in-hand with affected stakeholders. Ideally, affected communities, government agencies, NGOs, private sector orgs, and subject matter experts work together to identify and weight the vulnerability indicators so that the resulting vulnerability index both feels like an accurate reflection of peoples' lived experiences and a useful tool for guiding discussions around prioritizing funding and interventions.

Adams A, Byron R, Maxwell B, Higgins S, Eggers M, Byron L, Whitlock C. 2021. Climate change and human health in Montana: a special report of the Montana Climate Assessment. Bozeman MT: Montana State University, Institute on Ecosystems, Center for American Indian and Rural Health Equity. 216 p. https:// doi.org/10.15788/c2h22021.

Sevachandran G, Quek SG, Smarandache F, Broumi S. 2018. An extended technique for order preference by similarity to an ideal solution (TOPSIS) with maximizing deviation method based on integrates weight measure for single-valued neutrosophic sets. Symmetry 10:236. doi:10.3390/sym10070236.

Union of Concerned Scientists. (2019).   Killer heat in the United States: Climate choices and the future of dangerously hot days. 

Whitlock C, Cross W, Maxwell B, Silverman N, Wade AA. 2017. 2017 Montana Climate Assessment. Bozeman and Missoula MT: Montana State University and University of Montana, Montana Institute on Ecosystems. 318 p. doi:10.15788/m2ww8w.