During a pandemic, like that of COVID-19, it might be wondered how the number of cases is determined and how the lethality of a disease is determined. Some might be concerned or skeptical because the numbers often change over time and they usually vary across countries, age groups, ethnicities and economic classes. This essay provides a basic overview of a core method of making inferences from samples to entire populations, what philosophers call the inductive generalization.

An inductive generalization is an inductive argument. In philosophy, an argument consists of premises and one conclusion. The premises are the reasons or evidence being offered to support the conclusion, which is the claim being argued for.  Philosophers often divide arguments into inductive and deductive. In philosophy a deductive argument is such that the premises provide (or are supposed to provide) complete support for the conclusion. An inductive argument is an argument such that the premises provide (or are supposed to provide) some degree of support (but less than complete support) for the conclusion.  If the premises of an inductive argument support the conclusion adequately (or better) it is a strong argument. It is such that if the premises are true, the conclusion is likely to be true. If a strong inductive argument has all true premises, it is sometimes referred to as being cogent.

One feature of inductive logic is that a strong inductive argument can have a false conclusion even when all the premises are true. This is because of what is known as the inductive leap: the conclusion always goes beyond the premises. This can also be put in terms of drawing a conclusion from what has been observed to what has not been observed. The now dead David Hume argued back in the 1700s that this meant we could never be sure about inductive reasoning and later philosophers called this the problem of induction. In practical terms, this means that even if we use perfect inductive reasoning using premises that are certain, our conclusion can still be false. But induction is often the only option, and we use it because we must. So, when the initial numbers about COVID-19 turned out to be wrong, this is exactly what we should expect. The same must be expected in the next pandemic.

What, then, is an inductive generalization? Roughly put, it is an argument in which a conclusion about an entire population is based on evidence from a sample of observed members of that population. The formal version looks like this:

                                    Premise 1: P% of observed Xs are Ys.

                                    Conclusion: P% of all Xs are Ys.

The observed Xs would be the sample and all the Xs would be the target population. As an example, if someone wanted to know the mortality rate during a pandemic for males over sixty, the target population would be all males over sixty.

While the argument is simple, sorting out when a generalization is strong can be challenging. Without getting into the statistics and methods for doing rigorous generalizations, I will go over the basic method of assessment—so you can make some sense when experts talk about such matters during the next pandemic.

There will be various factors whose presence or absence in the sample can affect the presence or absence of the property the argument is concerned with, so a representative sample will have those factors in proportion to the target population. For example, if we wanted to determine the infection rate for all people, then we would need to try to ensure that our sample included all factors affecting the infection rate and our sample would need to mirror our target population in terms of age, ethnicity, base health, and all other relevant features. Sorting out what factors are relevant can be challenging, especially as a pandemic is unfolding. To the degree that the sample mirrors the target population properly, it would be representative.

A sample is biased relative to a factor to the extent that the factor is not present in the sample in the same proportion as in the population. This sort of sample bias was a problem when trying to generalize about COVID-19. One example of this was trying to draw a conclusion about the lethality of COVID-19. While the math to do this is easy (a simple calculation of the percentage of the infected who die from it) getting the numbers right is hard because we needed to know how people were infected and how many died from it.

Experts tried to determine the number of people infected by testing and modeling, which are also inductive reasoning. In the United States, most of the testing was of people showing symptoms, and this created a biased sample—to get an unbiased sample, even those without symptoms should be tested.  There was also the practical matter of the accuracy of the tests and the determination of the cause of death. This will be true in the next pandemic as well.

To use a concrete, but made up example, if 5% of those who tested positive for COVID-19 ended up dying, the generalization from that sample to the whole population would only be as strong as the representativeness of the sample. If only sick people were tested, the sample will not be representative and the conclusion about the lethality of the virus will (probably) be wrong.

There is also the challenge of sorting out the effect of the virus on different populations. While there is an overall infection rate and lethality rate for the whole population, there are different infection rates and lethality rates for different groups within the human population. As an example, the elderly were more likely to die of COVID-19 than younger people.

In addition to representativeness, sample size is important; the larger, the better.  This brings us to two more concepts: Margin of error and confidence level. A margin of error is a range of percentage points within which the conclusion of inductive generalization falls; this number is usually presented in terms of being plus or minus. The margin of error depends on sample size and the confidence level of the argument. The confidence level is typically presented as a number and represents the percentage of arguments like the one in question that have a true conclusion.

When generalizing about large (1o,000+) populations, a sample will need to have 1,000+ individuals to be representative (assuming the sample is taken properly). This table, from Moore & Parker’s Critical Thinking text, shows the connection between sample size and error margin (confidence level of 95%:

The practical takeaway is that sample size is important: a small sample will have a large margin of error that can make it useless. For example, suppose that a group of 50 COVID-19 patients received hydroxychloroquine tablets and 10 of them recovered fully. Laying aside all causal reasoning (which would be a huge mistake) the best we could say is that 20% of patients treated with hydroxychloroquine +/-14% will recover fully. This is just a simple generalization and a controlled experiment or study would be needed to properly assess a causal claim.

There are various fallacies (mistakes in reasoning) that can occur with a generalization. I will discuss those in the next essay. Stay safe and I will see you in the future.