My critical thinking class covers credibility, experiments and studies. As critical thinking is often seen as dull, I use real-world examples that might be marginally interesting to students. One is John Bohannon’s detailed account of how he “fooled millions into thinking chocolate helps weight loss.”

Bohannon’s con provides an excellent cautionary tale for critical thinkers. First, he shows how easy it is to rig an experiment to get “significant” results. As I point out to my students, a small experiment or study can easily generate results that seem significant. This is why it is important to have an adequate sample size. What is also needed is proper control, proper selection of the groups, and so on.

Second, he provides a clear example of a blight on academic publishing, namely “pay to publish” journals that lack peer review. While bad science does slip through peer review, these journals seem to publish almost anything, provided payment is made. Since the journals have reputable sounding names and most people do not know which journals are credible, it is easy to create a journal publication that seems credible. This is why I cover the importance of checking sources in my class.

Third, he detailed how news outlets published or posted the story without even perfunctory efforts to check its credibility. I cover the media in my class both from the standpoint of being a good journalist and being a critical consumer of news. I stress the importance of confirming credibility before accepting claims, especially when doing so is (supposed to be) one’s job.

While Bohannon’s con is evidence of problems with corrupt journals, uncritical reporting and consumer credulity, it raises other points worth considering. One is that while he might have “fooled millions” of people, he seems to have fooled relative few journalists (13 out of about 5,000 reporters who subscribe to the Newswise feed Bohannon used) and these seem to be more of the likes of the Huffington Post and Cosmopolitan as opposed to what might be seen as more serious health news sources. While I do not know why the other reporters did not run the story, it is worth considering that some did look at it critically. In any case, the fact that a few reporters fell for a dubious story is hardly shocking. It is, in fact, just what would be expected given the history of journalism.

Another point of concern is the ethics of engaging in such a con. It can be argued that Bohannon acted ethically. One way to do this is to note that using deceit to expose a problem can be justified on utilitarian grounds. For example, it seems morally acceptable for a journalist or police officer to use deceit and go undercover to expose harmful criminal activity. As such, Bohannon could contend that his con was effectively an undercover operation. He and his fellows pretended to be the bad guys to expose a problem and thus his deceit was morally justified.

One obvious objection is that Bohannon’s deceit did not just expose corrupt journals and incautious reporters. It also misinformed people. To be fair, any harm would be minimal. At worst, people who believed the story would consume dark chocolate and this is not a health hazard. Interestingly, as I am writing this, the view is that dark chocolate is beneficial. However, intentionally spreading such misinformation is morally problematic, especially since story retractions or corrections get far less attention than the original story.

One way to counter this objection is to draw an analogy to the exposure of flaws by “white hat” hackers. These hackers reveal vulnerabilities in software with the professed intent of forcing companies to address them. Exposing vulnerabilities can do some harm by informing the bad guys, but the usual argument is that this is outweighed by the good done when the vulnerability is fixed. Assuming it gets fixed.

While this does have some appeal, there is the concern that the harm might not outweigh the good. In Bohannon’s case it could be argued that he did more harm than good. After all, it was well-established that the “pay to publish” journals are corrupt, that there are incautious journalists and credulous consumers. As such, Bohannon did not expose anything new and merely added more misinformation to the pile.

It could be countered that although these problems are well known, it does help to continue to bring them to the attention of the public. Going back to the analogy of software vulnerabilities, it could be argued that if a vulnerability is exposed, but nothing is done to patch it, then the problem should be brought up until it is fixed, “for it is the doom of men that they forget.” Bohannon brought these problems into the spotlight and this might do more good than harm. If so, then this con would be morally acceptable—at least on utilitarian grounds.