In the last essay we looked at the inductive generalization and its usefulness in reasoning about certain aspects of the pandemic. As with all reasoning, one must be careful to avoid mistakes in logic—what philosophers call fallacies. Three fallacies often arise from efforts to generalize. These are the hasty generalization, appeal to anecdotal evidence, and biased generalization. I will look at each of them in terms of the current pandemic.
A hasty generalization occurs when a person draws a conclusion about a population based on a sample that is not large enough to adequately support the concussion. It has the following form:
Premise 1: Sample S (which is too small) is taken from population P.
Premise 2: In Sample S X% of the observed A’s are B’s.
Conclusion: X% of all A’s are B’s in Population P.
In the previous essay we saw a rough guide to sample sizes and looked at the margin of error and confidence level. In that context, the fallacy would occur when the sample was not large enough to warrant a person’s confidence in the conclusion. In the case of COVID-19 a generalization that is of great concern is drawing an inference about the lethality of the virus. The math for this is easy—the challenge is getting the right information.
At this time, there are large samples of infected people—thousands of people have been tested around the world. As such, the inferences from these large samples to the lethality of the virus would not be a hasty generalization. But avoiding this fallacy does not mean that the generalization is a good one—there are other things that can go wrong.
There are also inferences being drawn from relatively small samples, such as generalizations from various treatments being tested. For example, samples of people treated with hydroxychloroquine for COVID-19 are relatively small, so inferences from these samples to the whole population run the risk of committing a hasty generalization. This is not to deny that even small samples might have some use—in fact, we will talk more about this in later essays about causal reasoning. Speaking of causal reasoning, generalizations about causal factors also involves using causal reasoning, something that adds another level of complexity and induction.
As a practical guide, when you hear claims about the pandemic (or anything) based on generalizations you need to consider whether the conclusion is supported by a sample that is big enough. This does not mean that a sample that is too small means the conclusion is false—but it does mean that the conclusion is not adequately supported. A popular fallacy that is related to the hasty generalization is the appeal to anecdotal evidence.
An appeal to anecdotal evidence is committed when a person draws a conclusion about a population based on an anecdote (a story) about one or very few cases. The fallacy is also committed when someone rejects reasonable statistical data supporting a claim in favor of a single example or small number of examples that go against the claim. There are two forms for this fallacy:
Form One
Premise 1: Anecdote A is told about a member M (or small number of members) of Population P.
Premise 2: Anecdote M says that M is (or is not) C.
Conclusion: Therefore, C is (or is not) true of Population P.
Form Two
Premise 1: Good statistical evidence exists for general claim C.
Premise 2: Anecdote A is presented that is an exception to or goes against general claim C.
Conclusion: C is false.
This fallacy is like hasty generalization in that an inference is drawn from a sample that is to small to support the conclusion. The main difference between hasty generalization and anecdotal evidence is that the fallacy of anecdotal evidence involves using a story (anecdote) as the sample.
People often fall victim to this fallacy because anecdotes have more psychological force than statistical data. Wanting an anecdote to be true also tends to fuel this fallacy. As of this writing there has been no rigorously proven method of curing COVID-19 but there are various anecdotes about various methods purported to work. Even if these anecdotes are not lies, they do not provide an adequate basis for drawing conclusions about the general population. This is because the sample is not large enough to warrant the conclusion. As a concrete example, while there were some early positive anecdotes about hydroxychloroquine. Wishful thinking and Trump’s backing of the drug as a COVID-19 treatment caused some people to accept the anecdotes as adequate evidence—but this was bad reasoning.
Appeals to anecdotal evidence often occur in the context of causal reasoning, such as the case of hydroxychloroquine, and this adds in additional complexities. In a future essay we will revisit this fallacy in the context of causation in populations.
As with any fallacy, it does not follow that the conclusion of an appeal to anecdotal evidence is false. The error is accepting the conclusion based on inadequate evidence, not in making a false claim. It is also worth noting that anecdotal evidence can be useful for possible additional investigation—but is not enough to prove a general claim.
As noted earlier, there are large samples of infected people that allow generalizations to be drawn without committing the fallacy of hasty generalization. But even large samples can be problematic. This is because samples need to be both large enough and representative enough. This takes us to the fallacy of biased generalization.
This fallacy is committed when a person draws a conclusion about a population based on a sample that is biased to a degree or in a way that prevents it from adequately supporting the conclusion.
Premise 1: Sample S (which is too biased) is taken from population P.
Premise 2: In Sample S X% of the observed A’s are B’s.
Conclusion: X% of all A’s are B’s in Population P.
The problem with a biased sample is that it does not represent the population adequately and so does not adequately justify drawing the general conclusion. This is because a biased sample can differ in relevant ways from the population that impact the percentages of A’s that are B’s.
In the case of COVID-19 there is a serious problem with biased samples, although the situation is improving. I will focus on an inductive generalization about the lethality of the virus.
As mentioned many times, the math for calculating lethality is easy—the main challenge is sorting out how many people are infected. Since the start of the pandemic, the United States inflicted on itself a shortage of test kits. Because of this, many of the available tests (that are not being given to celebrities and politicians) have been used on people showing symptoms or who were exposed to those known to be infected. As of this writing nationwide general testing is not being done. As such, this sample is large but biased: it will contain a disproportionate number of people who are already showing symptoms and miss many people who are (or were) infected but asymptomatic. The likely result of this is a sample with a lethality rate higher than the real lethality rate of the virus. To use a simplified and fictional example: imagine a population of 1000 people and 200 of them are infected with a virus. Of the 200 people infected, 20 show symptoms and only they are tested. Of the 20 people tested, 2 die. This sample would show a mortality rate of 10%. But the actual mortality rate would be 2 in 200 which is 1 %. This would still be bad, but not as bad as the biased sample would indicate. This shows one of the many reasons why broad testing is important: it is critical to establishing an accurate lethality rate. An accurate lethality rate is essential to making rational decisions about our response to the pandemic, but we do not have that yet—and this ignorance is largely self-inflicted.
As a final point, it is also important to remember that the lethality varies between groups in the overall population—we know this based on the death data. But to determine the lethality for each group, the samples used for the calculation must be representative of the population. While the overall lethality is important, making rational decisions also requires knowing the lethality for various groups. For example, the virus seems especially lethal for the elderly and this should guide rational policies for protecting them.
As always, stay safe and I will see you in the future.
Leave a Reply