Offending the Analyst, the view "if you need to use statistics, then you have not designed your experiment very well".

[NNadir]

I download way more papers than I can possibly read, and download rapidly based on my mood or a thought that nags at me, thinking I'll read in detail after the library session. As a result, I have these rather large unreviewed directories, each corresponding to a particular year, on my computer filled with papers I never got to actually reading but collected.

In a sense, these lists represent a kind of diary related to what I was thinking about on a particular day; I haven't kept a real diary since I was 12 years old, a few centuries ago.

Here's a sample of my 2022 collections in January of that year:



Every once in a while, I go back and open a few of the papers that I didn't find time to read, and find them interesting.

Earlier today I noted over in the E&E forum a poster notable for attempting to greenwash fossil fuels as "hydrogen," carrying on insipidly about the old shibboleths of the fossil fuel industry, the elevation of purported nuclear risks including the usual poor science, and the claim that if anyone anywhere can be imagined to have died from exposure to radiation, either by specious association or otherwise, it is therefore OK for millions of people to die each year from dangerous fossil fuel waste.

As usual, this is intellectual and moral tripe, not worth too much of the carbon dioxide released to run a computer to comment on it.

In my opinion the fossil fuel industry, including those attempting to rebrand its products as "hydrogen," despises the nuclear industry because it is effectively the only industry that theoretically, if not practically, drive them out of business. Of course fossil fuel apologists need to make a mountain over a mole hill.

Anyway, tonight for some reason I opened by 2022 collection and came across a paper on the subject of studies of the Chernobyl Exclusion Zone, which at the time of the paper's publication, the authors had been studying for some 25 years, much of it on site.

The paper is this one:

N.A. Beresford, E.M. Scott, D. Copplestone, Field effects studies in the Chernobyl Exclusion Zone: Lessons to be learnt, Journal of Environmental Radioactivity, Volume 211, 2020, 105893.

The authors discuss, rather elegantly and fairly in my opinion the discrepancy among findings, ranging from little or no effect on living things to dramatic effects, at least on the level of molecular biology as well as population findings. Irrespective of these findings, there is a thriving ecosystem in the Chernobyl Exclusion Zone "CEZ," although we should not be surprised that it is a different ecosystem than the one present when the reactor failed as a result of a combination of design and operation.

The paper is open sourced; anyone can read it.

The purpose of my post however has not much to do with the excellent commentary therein on how to evaluate Chernobyl findings, but rather this remark that offends me a little:

We are familiar with the view “if you need to use statistics, then you have not designed your experiment very well”.

I'm not familiar with the view, indeed, without statistical analysis, particularly with respect to precision, measurement has no value. All experiments have a limit of reproducibility; all measurements are subject to random error and, in fact (a point the paper makes) selection bias.

In my view, an experiment is well designed if it does, in fact, include a statistical analysis.

I guess I'm a little prickly this evening, because the remark bugs me somehow.

[erronis]

1. My first thought is that the authors were referencing the use of the P-value

or various confidence intervals when a paper tries to convey some "significance" to the findings.

I agree that all scientific papers should include the statistics but much better, the actual data used to get those statistics.

On another note - I also am a voracious hoarder of articles that I think might be of some interest in one of my future lives. (I used to think one of my children would have an interest in what I do, but I've been dissuaded.)

I do find that some of the open-source search tools such as recoll, docfetcher, etc. are great for looking up documents in all of my scattered folders, whether from PDFs, emails/attachments. With some subtle crafting of a search query I have found an awful lot of useful nuggets amongst all the rest of my fool's gold.

[NNadir]

3. I have always used a "shotgun" approach to literature. I'm so old that it used to involve bringing loads of...

...quarters to the library, coming home with reams of paper.

I recall a certain amount of sadness when my wife prevailed on me to discard my paper collections of journals to which I used to subscribe: I had about ten years of the Journal of Organic Chemistry on my bookshelves.

If what I have on my little disk would easily fill a McMansion if it were paper.

[Bernardo de La Paz]

2. There is a difference between "need" and "use"

I think the point is that a goal is to have the design tease out the effect under study so that it is so obvious you don't need statistics, and then prove how obvious it is with statistics.

Not need statistics but use statistics to make it unassailable.

You can combine copper sulphate solution with sodium carbonate solution and get a precipitate 20 out of 20 times. You don't need statistics to show that it produces a precipitate, although you can call 20 out of 20 a statistic if you like.