As part of my critical thinking class, I cover the usual topics of credibility and experiments/studies. Since people often find critical thinking a dull subject, I regularly look for real-world examples that might be marginally interesting to students. As such, I was intrigued by John Bohannon’s detailed account of how he “fooled millions into thinking chocolate helps weight loss.”
Bohannon’s con provides an excellent cautionary tale for critical thinkers. First, he lays out in detail how easy it is to rig an experiment to get (apparently) significant results. As I point out to my students, a small experiment or study can generate results that seem significant, but really are not. This is why it is important to have an adequate sample size—as a starter. What is also needed is proper control, proper selection of the groups, and so on.
Second, he provides a clear example of a disgraceful stain on academic publishing, namely “pay to publish” journals that do not engage in legitimate peer review. While some bad science does slip through peer review, these journals apparently publish almost anything—provided that the fee is paid. Since the journals have reputable sounding names and most people do not know which journals are credible and which are not, it is rather easy to generate a credible seeming journal publication. This is why I cover the importance of checking sources in my class.
Third, he details how various news outlets published or posted the story without making even perfunctory efforts to check its credibility. Not surprisingly, I also cover the media in my class both from the standpoint of being a journalist and being a consumer of news. I stress the importance of confirming credibility before accepting claims—especially when doing so is one’s job.
While Bohannon’s con does provide clear evidence of problems in regards to corrupt journals, uncritical reporting and consumer credulity, the situation does raise some points worth considering. One is that while he might have “fooled millions” of people, he seems to have fooled relative few journalists (13 out of about 5,000 reporters who subscribe to the Newswise feed Bohannon used) and these seem to be more of the likes of the Huffington Post and Cosmopolitan as opposed to what might be regarded as more serious health news sources. While it is not known why the other reporters did not run the story, it is worth considering that some of them did look at it critically and rejected it. In any case, the fact that a small number of reporters fell for a dubious story is hardly shocking. It is, in fact, just what would be expected given the long history of journalism.
Another point of concern is the ethics of engaging in such a con. It is possible to argue that Bohannon acted ethically. One way to do this is to note that using deceit to expose a problem can be justified on utilitarian grounds. For example, it seems morally acceptable for a journalist or police officer to use deceit and go undercover to expose criminal activity. As such, Bohannon could contend that his con was effectively an undercover operation—he and his fellows pretended to be the bad guys to expose a problem and thus his deceit was morally justified by the fact that it exposed problems.
One obvious objection to this is that Bohannon’s deceit did not just expose corrupt journals and incautious reporters. It also misinformed the audience who read or saw the stories. To be fair, the harm would certainly be fairly minimal—at worst, people who believed the story would consume dark chocolate and this is not exactly a health hazard. However, intentionally spreading such misinformation seems morally problematic—especially since story retractions or corrections tend to get far less attention than the original story.
One way to counter this objection is to draw an analogy to the exposure of flaws by hackers. These hackers reveal vulnerabilities in software with the stated intent of forcing companies to address the vulnerabilities. Exposing such vulnerabilities can do some harm by informing the bad guys, but the usual argument is that this is outweighed by the good done when the vulnerability is fixed.
While this does have some appeal, there is the concern that the harm done might not outweigh the good done. In Bohannon’s case it could be argued that he has done more harm than good. After all, it is already well-established that the “pay to publish” journals are corrupt, that there are incautious journalists and credulous consumers. As such, Bohannon has not exposed anything new—he has merely added more misinformation to the pile.
It could be countered that although these problems are well known, it does help to continue to bring them to the attention of the public. Going back to the analogy of software vulnerabilities, it could be argued that if a vulnerability is exposed, but nothing is done to patch it, then the problem should be brought up until it is fixed, “for it is the doom of men that they forget.” Bohannon has certainly brought these problems into the spotlight and this might do more good than harm. If so, then this con would be morally acceptable—at least on utilitarian grounds.
“Second, he provides a clear example of a disgraceful stain on academic publishing, namely “pay to publish” journals that do not engage in legitimate peer review.”
The US government is pushing for “open access” to research articles. These open access journals often charge a lot of money for publishing a paper.
Scientists are busy and are asked to review a lot of papers without compensation. “Peer review” is typically only a quick read and a thumbs up or thumbs down.
I’m fine with fees that cover the cost of publication (although no journal I have been published in charged me, though one is selling an article I wrote with a physicist on Amazon). What I am against is journals that publish just because a person says.
Most professionals are busy, but peer review is rather important to helping maintain the integrity of published research. Folks I know in the sciences routinely do uncompensated peer review (as have I). But, as you say, some people are rather slack in their reviews.
One solution might be to use some of the fees people pay for publication go to compensate reviewers (who would be paid for a good review, whether the article is published or not). Of course, they would need to take steps to prevent exploitation of the system.
peer review is rather important to helping maintain the integrity of published research.
Peer review is grossly overrated. The only peer worthy of the title is reality. Build it. Prove it. Describe how it was done so an independent entity can do the same and verify. If the subject is too complex, break it down into smaller parts. Build them, prove them. Repeat. All else is false confidence.
Ran across this today. In the interest of beating a dead horse:
“A foolish faith in authority is the worst enemy of truth” Albert Einstein
Einstein, you guys. The ultimate earthly authority. Until Niels Bohr’s peer review. But why believe him? ‘Cause you know, Einstein.
“Most professionals are busy, but peer review is rather important to helping maintain the integrity of published research.”
Yes and no. Peer review mainly serves to filter out obviously flawed research. It is also used to help determine which research gets published in the glossy journals.
True-there are connected networks in academics. Everyone in the game eventually learns that only certain people get to publish in the “top” journals.
Here is some red meat for Magus and WTP: what restroom should Caitlin Jenner use?
I really haven’t been following too closely. Does he/she/xe/xir/ze/zir/it stand up or sit down? Or is that question not permitted?
Permitted, but irrelevant: http://kk.org/cooltools/archives/24145
Yeah..they’d only make us pay for those stupid devices, send teenage boys to the store to buy boxes of such for them, and then try to flush the damn things.
I’m slowly coming around to the idea that all bathrooms be unisex (just like they have in Europe!…or so I was told as a youngster). Equality and all that. Really. What is the use case for having two completely different sets of facilities to facilitate the same functionality?
The one for reality TV show stars and self-promoters.
Bannon was probably seeking attention and publicity via shaming the media. Shaming is all the rage today. Alan Sokal, in 1996, was probably the first person to submit a gibberish paper and have it published in an academic journal.
Bannon was probably seeking attention and publicity via shaming the media. Shaming is all the rage today
Because the shame is long overdue. The media have been shaming via “gotcha” journalism and such for decades now. All the while being guilty of the same sort of sins. But in the same vein of “who guards the guards, who polices the police?” who reports on the reporters? Academia is not far behind. People who live in glass houses and such.
Some journals do publish absurd crap. I’ve also heard papers at conferences that were absurdly horrible, yet praised. I’m counting on WTP to take this opportunity to take some shots at me. Don’t disappoint.
I think it’s a logical fallacy to write nonsense smothered in intellectual jargon that makes the nonsense seem like it must be saying something profound if people are smart enough to understand it. I think a lot of philosophy books, and philosophers, are like this fallacy: full of seemingly intellectually crap. 😛
I’m counting on WTP to take this opportunity to take some shots at me
Nah. I’m good. What AJ said.
I’m worried about you. Have you been eating enough?
Don’t cry for me ‘Tina. I had the usual capitalist breakfast, two cuddly kittens with a slice of hobo bacon. I’d be more concerned about how you plan to survive if you manage to outlive this education bubble. By the time this disaster blows, I’m sure the robots will have taken over McD’s and similar.
Not to worry. No one survives when Google kills us all.
Google? Don’t be silly. After all (TM) their corporate motto is “Don’t be evil”. Of course the road to hell is paved with good intentions. Or so said some “smart” people. At least one of whom was the inspiration for the widest spreading of hell on earth in human, and even non-human, history.
Smart evil convinces people it is good. Stupid evil lets people know it is evil.
It is becoming a cliche that the devil’s greatest trick is convincing the world he doesn’t exist. Best source I could find is from the 90’s movie The Usual Suspects, though I have an inkling I heard it before that. I would say he has a trick that gets a little less publicity, and that is casting Good as Evil. Taking opportunity to cast Good’s failings as evil intent. Sometimes he even dresses up in the clothes of Good to do his evil.
To me, Good is truth. Evil dodges truth, or as I like to call it, reality. It dodges truth first with lies. If lies doesn’t satisfy evil it moves on to deception then theft, numerous other vices and ultimately, because evil refuses to accept reality/truth, it resorts to murder.
But we can play games with words and allusions and other deceptions for eternity and never get anywhere. Let’s play a different game, Mike. I’ll pick three things that exist in the real world and rank them from good to evil. Agree/rearrange as you see it, then you give me three things you pick in good-to-evil order and I’ll do the same. Want to play? As I said, I’ll go first:
The typical cop
Google
Black “Leadership” of Al Sharpton, et al.
Ball’s in your court, cowboy.
I’ll play. The typical cop helps hundreds. Google helps billions.
Google
The typical cop
Black “Leadership” of Al Sharpton, et al.
From the final Wikipedia Entry for Google:
“On April 1, 2115 the Google Net announced that it had completed its utilitarian calculation and determined that humans created more evil than good. All humans in Google Autonomous Vehicles were taken to rendering sites and converted to raw materials. Humans in Google Buildings were rendered on site. Using Amazon delivery drones and its own robots, Google tracked down the few remaining humans and they, too, were rendered. Google is good. Google is all.”
Ah TJ, but surely the typical cop, unlike Google, lacks the desire or capacity to “kill us all”. Of course one could argue that “we” are all evil relative to Google and thus were Google to “kill us all”, it would be doing good and not evil. An interesting perspective. But then one would be admitting to being evil. And that would be stupid. The admitting, I mean.
And there you have it, TJ. Google is evil because … well because. Try to drive the discussion to a concrete reality and out comes the clown nose and he runs away.
You do understand what I’ve been trying to tell you for years now, yes? Argument is futile when the only universe he will play in is one of his own construction. For a guy who flat out admits how he really hates to lose (I mean, who doesn’t?) it’s a safe strategy. If you can call it strategy.
Wow.
How wow?
I was thinking about your article last night, because I saw a media report about CERN and the LHC being fired up again, and I thought “The only thing worse than media reporting the bogus science of Bohannon’s con as fact is media reporting the bogus science of CERN and the LHC con as fact, because the physics establishment hasn’t yet acknowledged this con and media isn’t questioning it.”
BOOK – The Higgs Fake: How Particle Physicists Fooled the Nobel Committee, by Alexander Unzicker – http://www.amazon.com/The-Higgs-Fake-Physicists-Committee/dp/1492176249
VIDEO – Alexander Unzicker at the 2nd Rational Physics Conference in Salzburg-Austria: https://youtu.be/hNg4cCe6A74 via @YouTube