What’s the problem? Intrafish published an excellent commentary from Kristin Langeland and Krister Hoaas from Sjømat Norge (Seafood Norway) asking whether models are sufficiently accurate to be used as the basis for regulation of individual farms. Their commentary was prompted by the decision of the Scottish appeals body to reject SEPA’s changes to salmon farm licences imposed by the regulator SEPA. They ask whether the Scottish appeals decision is also relevant to Norway because the Scottish appeals body raised the question of modelling as a way of assessing the impact of sea lice on wild salmon.
However, what attracted my attention most from their commentary was they repeated the Storting’s order for “a more comprehensive and thorough knowledge base for the environmental impact of aquaculture including the industry’s actual and relative impact on the Norwegian wild salmon population.”
They also add that a broad majority in the Norwegian Parliament (Storting) also set guidelines that a “broadly composed selection of researchers and professionals should participate in this work and there should continue to be transparency about methods and models.”
The big question is whether any attempt is being made to fulfil the Storting’s order or are their instructions simply being ignored. If I remember correctly, the Norwegian Government requested the Institute of Marine Research (IMR) for better knowledge, but they are unlikely to challenge existing knowledge since IMR have been at the leading edge of the current narrative.
It is unclear what the Storting mean by a selection of researchers and professionals but there are certainly scientists working in the industry who could contribute to the knowledge base as well as other scientists who have a more general knowledge. The Seafood Research Fund (FHF) have advertised their uncertainties project, but any outcome could be more than three years away.
My view is that any changes to the regulation will ultimately affect the salmon farming industry so clearly if no other option is available, then the industry should take up the baton and initiate its own search for better knowledge. It isn’t necessary to work for one of the research institutions to understand the relationship between sea lice, salmon farming and wild fish. To some extent I wonder if working for an institution is actually a liability. For example, a recent paper about the decline of wild salmon in Norway that concluded that the decline was primarily driven by reduced marine growth but with regional variation caused by a number of different man-made impacts. What is interesting about this paper is that it had fifteen authors from eight different research institutions. In my opinion, such collaboration reduces individual thinking. Many papers about sea lice are written by multiple authors following a similar narrative. Any individual thought is usually supressed if it doesn’t fit the narrative, which is exactly why the salmon industry is exposed to such pointless regulation. It is only necessary to consider the Expert Group seminar held earlier this year in Bergen. They only wanted to hear new knowledge and gave an explicit instruction to this effect. It is difficult to understand why? What is the problem about discussing the existing science to ensure that everyone is happy with it. I would suggest that there is so much uncertainty about the science, (enough that FHF are prepared to spend NOK 20 million on a project to understand the uncertainties) yet no-one from Government, the scientific establishment or angling organisations are prepared to discuss the subject, mostly claiming the science is robust. Yet NOK 20 million (£1.5 million) seems a lot to spend on reviewing such robust science. I would argue that what actually is required is to take the science back to first principles with a first question as to whether sea lice associated with salmon farms are actually responsible for the decline in wild fish stocks, not whether the models say they are.
In a previous reLAKSation I mentioned how Fisheries Management Scotland had said that they were disappointed about the Scottish appeals decision. Now the Missing Salmon Alliance including the Atlantic Salmon trust have followed suit. They say two key issues remain unequivocal – the science is robust and the need for regulation is urgent. I would disagree on both points, but what is of most interest is that they say that “members of the MSA offer to engage constructively with government and stakeholders on this issue”
I’m willing to bet that they refuse to engage with me.
Small is not beautiful: Back in 1973, the book charts were dominated by a book written by E.F Schumacher entitled ‘Small is Beautiful – A study of economics as if people mattered’. It was written in response to a mainstream ethos of bigger is better. I am now indebted to Kristin Langeland and Krister Hoass of Sjømat Norge for their article in Intrafish which mentions another economics tome by Nobel Prize in Economics winner Danile Kahneman – Thinking, Fast and Slow. They refer to one chapter of the book – the law of small numbers. I now have a copy of the book, but Langeland and Hoaas summarise the crux of the matter so well, that I prefer to repeat what they said.
Kahneman shows that we humans have a strong tendency to believe that small samples resemble the population from which they are drawn just as well as large samples do. This is not the case. The smaller the sample, the more chance prevails, and the more often we get misleading results in both directions. We look for patterns and create coherent narratives. We notice the content of a finding but overlook how many observations it is based on. Researchers are not immune. Together with Amos Tversky, Kahneman found that experienced researchers, including statisticians, relied too much on results from small samples, and often chose samples so small that they risked overlooking real effects or finding effects that did not exist.
I have argued for several years that any assessment of the impact of seal lice associated with salmon farming relies too heavily on sample sizes that are far too small. This applies to both Scotland and Norway. In Scotland, there is a protocol that says that there should be a minimum sample size of 30 fish. I believe that this is still far too low and I think that the number 30 was selected because of its use in the Central Limit Theorem which states that the distribution of a sample approaches normal as the sample size grows and 30 is considered as the point when the approximation is reliable. However, there is a flaw to this idea as sea lice are not distributed as a normal distribution, so the 30 sample is actually invalid. As I have previously highlighted just 18% of the samples collected between 1997 and 2019 by the Fisheries Trusts achieved a sample size of 30 fish. I have also mentioned previously that Taranger (2012) suggests a sample size of 100 fish. In 2023 for example, the Traffic Light assessment for seven of the 13 POs was based on total samples of less than 100 fish.
The Sea Lice Expert Group report includes sampling in their list of uncertainties, and they say that the sampling is ‘adequate’. The question for the Norwegian Government is whether they think that just an adequate number of samples is sufficient on which to base decisions about the future of one of the country’s most important industries?
As I have pointed out, the Expert Group refuse to discuss sampling because it falls into existing knowledge, even though earlier this year IMR published a comment that to obtain a totally reliable estimate of sea lice infestation on salmon farms, 300 fish should be sampled. Surely, the same should apply to wild fish too, if not even more so.
What is clear is that the science of sampling is not robust. Whilst the scientific community appear unwilling to discuss the issue, I have been able to gain an insight into their thinking through the critique of my review that Marine Directorate scientists shared with four scientists from IMR. The issue of small samples is one of the six subjects on which they commented. They begin:
As a general problem, many samples collected in validation of sea lice impacts are smaller than sampling protocols would aim for. Unfortunately, sampling of wild salmonid populations is opportunistic and it is often impossible to obtain more than a few fish for a given sampling location, especially if that population is under stress.
What they appear to say is they recognise sampling wild fish is hit and miss, and mostly miss, leading to considerable doubt as to validity of the results. What this means is the decisions about the future of salmon farming are being made on the sea lice counts on just a few fish. They continue:
This is a problem because it means confidence intervals around sample estimates of the population sampled are large, particularly in the light of aggregated distributions. As a result there may be considerable uncertainty in population estimates of lice abundance made from a single sampling time at a single location. However, small samples can be amalgamated, they are then a sample of the population from which the samples are drawn and will incorporate the variation in the population that the samples are taken from.
The big question is whether one hundred samples of just one fish are the same as one sample of a hundred fish. I would argue not, especially as the difficulty in seine net sampling means that it could be that the fish caught are less able to evade capture possibly due to weakness from stress caused by infestation.
The critique continues that:
The author (me) questions whether samples can be amalgamated, yet all through the report uses samples that are amalgamated across Production Areas. It is true that by amalgamating samples you are increasing the number of sub-populations sampled and this is likely to increase drivers of deviation. This deviation can be driven due to exposure to times or locations of high infection and times or locations of lesser infection pressure, and so amalgamating samples does increase the population variance. This high variance in the graphs presented in the report is likely due to precisely the naïve approach to amalgamating the data that the author has used.
Unfortunately, the critique misses the point that I have used the data that is available and have in some instances combined all the data together in order to produce a graph to see if the data resembles an aggregated distribution or not. As the site data consists of relatively small samples, I had no choice except to adopt this approach otherwise the alternative was to reject all the data as invalid (which is likely what it is)
Small samples can also be used to assess patterns within populations, even where the individual sample has wide confidence intervals. Care must be taken when using small samples, and statistical significance of associations found may be underestimated. However, the more data available the tighter the confidence intervals. The bringing together of different results obtained in different studies is the principle used in Cochrane reviews, under which many studies are combined (sometimes with no certain results) to produce a meta-analysis with reduced confidence ranges.
However, just because Cochrane reviews amalgamate data, does not make it right for sea lice analysis. Finally, the critique included a couple of other comments that I found of interest. These are:
Small samples lead to reduced ability to detect effects that are real as absence of evidence is not evidence of absence.
The recommendations outlined in the Expert Group reports and Evaluation Committee report by Eliasen et al. (2021), which conclude that estimates of seal lice levels derived from the various sampling methods employed have limitations.
I actually disagree with the Scottish scientists because whilst they say that small samples reduce the ability to deflect effects that are real, small samples can also indicate effects that appear present but are not real. Instead of suggesting that the absence of evidence is not evidence of absence, the answer is simple – take much larger samples. This is also what the International Committee indicate – small samples have their limitations. That is exactly my point.
UDN: There have been multiple deaths of wild salmon in the Orkla, Gaula, Veralselva and Mossa rivers in Trøndelag in Norway. The likely cause is Ulcerated Dermal Necrosis (UDN) complicated by secondary infection of saprolegnia fungus. According to iLAKS, the Norwegian Veterinary Institute have been called in to investigate but can do little because the cause of the disease is unknown. There has been some conjecture as to whether the disease has been found in salmon farms or whether salmon farms might be at risk of infection from wild fish. The answer is no-one really knows, although one theory is stress from climate change.
iLAKS
However, there have been many comments left on angling sites that the underlying problem is caused by the presence of salmon farms. This is not unexpected as the anglers’ response to any issue is that it must be due to salmon farming.
What is known is that the last time that UDN was seen in Scotland was in the late 1960s and early 1970s and caused significant damage to wild fish stocks. This was well before salmon farming arrived in Scotland. Since then, UDN appears to have totally disappeared although secondary infections of saprolegnia do appear from time to time.
It is easy to forget that wild salmon are often subjected to disease pathogens that occur naturally. Since the 1970s, it isn’t the fault of salmon farms.