Wildfire Risk Models Evaluation with TOC Curves
Max Lutz, Ravi Thapaliya, Rahebe Abedi, Brenner Burkholder, Mina Wei and Sai Vishal
Objectives of this study
- We created three fire susceptibility maps: Base, NDVI difference, and Evaporative Stress Index (ESI). The Base model uses five static variables. The NDVI difference model combines an NDVI seasonal difference with the same five static variables as the Base model. The ESI model combines one ESI variable with the same five static variables as the Base model.
2. We used the Total Operating Characteristic to compare each ignition suseptability map to a map of fires during the year 2021.
Study Area
- This study looks at the state of Idaho for ignition susceptibility of wildfires.
- The central and northern regions are mountainous and forested, with the southern region being predominantly scrubland and agriculture
- Idaho has seasonal precipitation and temperature patterns, creating conditions for a distinct fire season from June until September.
Deductive Model Flow
- Inputs were separated into two categories Static and Non-Static. Static inputs are those that do not change depending on the year of study, for this study, that is the year 2021. Non-Static inputs are those that change depending on the year of study.
- The static variables in green were reclassified into ranks, such that higher ranks have a higher ignition potential, and summed to create the base model.
- The non-static NDVI difference and ESI were reclassified and ranked as well; being added to the base model to create the NDVI Difference Model and the ESI Model for the year 2021.
Historic Fires from 1950-2020 (Static)
- Historic fire density was calculated as the total burned area within a given watershed throughout the historical range divided by the total area of the watershed.
- Fire densities were expressed as a percent of the watershed leading to values greater than 1 due to the burn areas being summed over a 70-year time period, with recurring fires being common.
Rankings:
- 0 = 0%
- 1 = (0% to 50%)
- 2 = (50% to 100%)
- 3 = (100% to 573%)
Vegetation Type (Static)
- Vegetation was reclassified into 7 classes, ranking finer fuels as having higher ignition potential than vegetation types that had corser fuels.
- We Assumed non-vegetated land cannot burn.
Rankings:
- 0 = Non-Vegetated
- 1 = (No Dominant Fife Form and Sparcley Vegetated)
- 2 = (Open Tree Canopy and Closed Tree Canopy)
- 3 = Sparse Tree Canopy
- 4 = Dwarf-shrubland
- 5 = Shrubland
- 6 = (Herbaceous shrub-step and Herbaceous grassland)
Wildland Urban Interface (Static)
- Wildland Urban Interface (WUI) is split into three categories: Intermix WUI, Interface WUI, and non-WUI. Intermix WUI is where urban and vegetation intermingle, and where urban density is greater than 1 house per 40 acres with greater than 50% of the area in wildland vegetation. Interface WUI are settled areas that have less than 50% vegetation, but lie within 1.5 miles of a densely vegetated area that is at least 5 square kilometers in size. Non-WUI are areas that do not fit any of these definitions.
- The idea behind the use of WUI is that humans contribute to the ignition of wildfires, such that densely populated areas with large amounts of wildland are more susceptible than other locations.
Rankings:
- 0 = (Non-WUI)
- 1 = (Intermix)
- 2 = (Interface)
Slope (Static)
- Fires can spread faster and ignite more land given a higher slope. Slope was therefore classified assigning higher slopes to higher ranks.
Rankings:
- 1 = (0°to 10°)
- 2 = (10° to 20°)
- 3 = (20° to 90°)
Aspect (Static)
- Southern aspects get more sun than northern aspects, leading to drier conditions on south-facing slopes. While western aspects get dry wind from the mountainous region due to predominant westerly winds, as well as get evening sun which is more influential on evaporation than the morning sun.
Rankings:
- 0 = Flat
- 1 = North
- 2 = East
- 3 = South and West
Evaporative Stress Index (Non-Static)
- This index estimates the water that is lost from Evapotranspiration (ET).
- ESI shows areas of drought where vegetation is in stress conditions because of the deficiency of water, leading to its use for drought monitoring. We therefore assigned greater ignition potential in areas that were experiencing greater evaporative stress.
Rankings:
- 0 = (0 to -2.10)
- 1 = (-2.10 to -2.80)
- 2 = (-2.80 to -3.15)
- 3 = (-3.15 to -3.36)
- 4 = (-3.36 to -3.50)
NDVI Seasonal Difference (Non-Static)
- NDVI Seasonal Difference was calculated as (Growing Season NDVI) - (Fire Season NDVI), with the growing season being from March through May, represented as a mean, and the fire season being from June through September represented by a mean.
- Negative ranks were used due to the reduction of risk given negatives indicating active growth in the fire season compared to the growing season. A rank of zero indicates areas that did not change throughout the different seasons and therefore NDVI Difference cannot give higher or lower ignition potential for those regions.
Rankings:
- -4 = (-0.62 to -0.10)
- -3 = (-0.10 to -0.04)
- -2 = (-0.04 to -0.01)
- -1 = (-0.01 to 0.02)
- 0 = (0.02 to 0.06)
- 1 = (0.06 to 0.10)
- 2 = (0.10 to 0.15)
- 3 = (0.15 to 0.19)
- 4 = (0.19 to 0.87)
Base Model Output
Summation of
- Historic Fire Density
- Vegetation Type
- Wildland Urban Interface
- Slope
- Aspect
Below is a Total Operating Characteristic (TOC) curve for the base model, with the model output as the ranked variable, and the 2021 fire season as the boolean variable. A steeper slope between two thresholds indicates a higher hit intensity for the segment. The curve shows higher ranks near the origin experienced fire more intensively than lower ranks. The False Alarms equals Misses point on the curve is where the quantity of fire in the model matches the quantity of fire during 2021.
Base Model TOC curve
Evaporative Stress Index Model Output
Summation of
- Historic Fire Density
- Vegetation Type
- Wildland Urban Interface
- Slope
- Aspect
- ESI
Spatial coarsening in the ESI Model is due to the inclusion of ESI variable, which is much courser compared to the other inputs.
NDVI Seasonal Difference Model Output
Summation of
- Historic Fire Density
- Vegetation Type
- Wildland Urban Interface
- Slope
- Aspect
- NDVI Seasonal Difference
Multi-TOC Comparison
Below is a TOC space with all three models present to compare the TOC outputs from the base model, the ESI model, and the NDVI difference model. Below the multi-TOC comparison is a zoomed-in image of the same TOC space, looking closer at the origin in order to see the threshold labels. From looking at the TOC curves below one can see that the ESI model improved on the base model seen in the higher hit intensities at higher ranks and the false alarms = misses point being closer to the upper left bound of the TOC space. While the NDVI difference model detracted from the base model, seen in the shallower slopes between thresholds and a lower false alarms = misses point. The Area Under the Curve (AUC) is 0.627 for the base model, 0.527 for the NDVI seasonal difference model, and 0.733 for the ESI model.
Multi-TOC Comparison
Zoomed-in Multi-TOC Comparison
Conclusions
- The Incorporation of the ESI variable into the Base model increased the accuracy of the model. The ESI model improved the base model, with increased hit intensity of high-risk thresholds in the ESI model versus the base model. The ESI Model’s false alarms = misses point was closer to the upper left bound of the TOC area than the base model’s, indicating less allocation error.
- The incorporation of the NDVI seasonal difference variable into the Base model decreased the accuracy of the model. This is seen with lower hit intensity of higher thresholds in the NDVI Difference Model compared to the base model. The NDVI Difference Model’s false alarms = misses point is further from the upper left bound of the TOC area than the Base model or the ESI Model's, indicating more allocation error than the base model.
Learning More About the Total Operating Characteristic
If you would like to learn more about the Total Operating Characteristic (TOC) please make use of the book Metrics that Make a Difference by Robert Gilmore Pontius. As well as other implementations of the TOC such as Liu & Pontius 2021. If you would like to generate your own TOC Curves, please make use of Zhen Liu's TOC Curve Generator: https://lazygis.github.io/projects/TOCCurveGenerator