fishy numbers : an inquiry

If you know what the ADFG chart on the right means and how the numbers are derived, you can skip ahead using the navigation headers above.

In order to understand this issue, it is important to first get a sense of how the forecast numbers are derived. In this first section, I will do my best to explain some key pieces, but here are a few recent ADFG documents that can give you further background on the current status of ADFG herring research and that I'll be referencing along the way:

•  2019 Sitka Stock Assessment  (Methodology described in greater detail here)
•  Sitka Sound Herring Forecasts 2020 and 2021 — Dr. Sherri Dressel  (PDF 856 kB)
•  ADF&G Herring Survey and Sampling Results 2019 and 2020 — Kyle Hebert  (PDF 3,026 kB)

Sitka Sound is enjoying something of a herring boom right now thanks to the very-dominant cohort of herring hatched in 2016. Now in 2021 those herring are age-5 and there are a lot of them - in Sitka and up and down the coast.

To the right is the department's estimated biomass for 2021 at 210,453 tons, up steeply from the estimate of 64,343 two years ago. Inset is an indicator of the forecast percentage composition by age as derived from cast net samples, showing that 86% of the Sitka Sound herring are forecast to be age-5 (class of 2016) this year.

That biomass estimate isn't reached by any direct counting of herring. Instead, the centerpiece of the ADFG herring study is the annual estimate of egg deposition in Sitka Sound, which is arrived at by monitoring Sitka Sound for spawning via aerial surveys and then conducting dive samples from which to extrapolate spawn density for different sections, and ultimately the whole area.

In 2020, ADFG reported 23 trillion eggs in Sitka Sound- more than the department has ever registered before.

From that number, the estimated spawning biomass of herring for the year can be calculated. Towards answering the question of how many herring it takes to produce 23 trillion eggs, the Department needs to have age/sex/size samples that indicate the weight-at-age (size on average for each age-class), proportion-at-age (what proportion of the biomass consists of the different age classes), and also needs an established relationship between weight and fecundity.

The Department's forecasts are typically issued through one or more of three different methods of sampling herring each year: active spawning cast nets, commercial purse seine test sets in spring, and apurse seine test fishery in winter. The last time all three types were done in one year was 2016. You can see the form of the sample results on the chart on the right, which are published in the annual ADFG  Stock Assessment Survey .

This chart shows the average weight at age by sample type over the 2011-2016 time span. It is clear that the commercial samples are capturing data about prime fish rather than about population norms. One explanation for some of this difference is the presence of spawned out (and thus lighter) fish in the cast-net samples - but that doesn't reasonably account for the entire difference in sampling methods.

To the right is a primitive analysis of the average percentage-at-age for the three sampling methods during the years that the methods were used side by side.

The results are striking and even intuitive; the commercial fleet is looking for larger herring, and so their sample results are bound to be higher than population averages.

Note the easy avoidance in commercial samples of age-3 and age-4 fish in favor of older herring, age-8+ fish in particular.

This is also a temporal pattern - indigenous science has long recognized that older herring spawn first. Since the commercial fishery is interested in fishing before spawning has begun, those samples will naturally be skewed towards older herring. We can see this pattern in the active spawning cast net samples from 2019 and 2020 (data provided to me by ADFG via correspondence).

The chart on the right is derived from 2019 sample data - the solid blue line indicates the proportion of age 3+4 in the samples from early in the season (left) to later in the season (right); the solid orange line does the same herring aged 5 and older. Each line is paired with a trend-line.

As you can see, the samples yielded more young fish and fewer older fish as the season went on.

Here is a similar chart from the 2020 active spawning cast net samples showing a similar pattern.

Until recently, ADFG arrived at preseason weight-at-age estimates by balancing different sources, but quietly stopped doing winter test fisheries in 2016 and does not rely on cast net data for weight results, given the presence of spawned out fish. ADFG's current practice is to strongly privilege the results of samples from the commercial fishery, even though the data is more likely to misrepresent population structure and weight-at-age. By increasingly emphasizing sample sources that skew toward larger fish, are we getting a distorted analysis of age structure and disproportionate fishing of older fish?

ADFG is perfectly aware of the high-grading pattern present in the commercial sets, as you can see in this excerpt from the Stock Assessment Survey: "benefits of using data from cast net samples are that they provide a more complete and consistent time series and bias is expected to be lower than for fishery-dependent data that may be influenced by targeting larger fish."

To recap: ADFG is using outdated 2017/2018 commercial catch samples to guide forecasting and management decisions, even though there are 2019 and 2020 cast net samples available to help us understand this present herring run that has a notably unusual population structure.

What does that mean for the herring? Let's look at those 2019 and 2020 samples.

Pictured here is the result from the spring cast net survey in 2019 when ADFG first became aware of the dominant 2016 age class. The sample indicates that the new recruit class came in small but mighty, at 76% of the whole population but with the 7th smallest weight-at-age in departmental records. Meanwhile, the rest of the age spectrum also has weight-at-age at the very low end of historic measures.

ADFG hasn't publicly released the data for 2020 yet, but it has shared two charts, which can give us a good sense of things. This first chart shows that the proportion of class-of-2016 herring may actually be 90% of the biomass, up a little bit from prior estimates.

And from the final points on this chart that Kyle Hebert shared at the preseason meeting - currently the only source of data on 2020 herring - it looks like age 3 (from 2017) herring came in at maybe 62g last year, age 4 (2016) herring at 83g, age 5 (2015) at 96g, age 6 (2014) at 110g, age 7 (2013) at 109g, and age 8+ (2012 and prior) at 125g. This chart indicates that herring haven't been as small-at-age as they were last year since the early 1990's.

Now let's look at the 2020 forecast again, keeping in mind that it is the same forecast (derived from 2017 and 2018 commercial samples) that is being used this year.

This forecast was used last year and is being used again this year. Was it accurate in describing the herring encountered in 2020?

Let's see...