Spotting bullshit in the wild it isn't something you have to let others do for you. To illustrate this, we've provided a set of case studies based upon examples of bullshit in the wild. We've spotted many of these ourselves; some have come via other channels. These cases aren't the most egregious examples out there, but each illustrates one or more of the principles and practices that we aim to teach in this course. We will be adding additional case studies on a regular basis.
Track and field records as examples of senescence. We lead off our series of case studies by calling bullshit on a figure in one of our own publications. We explore how differences in sample sizes can create misleading patterns in data, and an example of how writing a simulation can be an effective method of calling bullshit.
Food stamp fraud. In this example drawn from a Fox News story, we demonstrate how Fermi estimation can cut through bullshit like a hot knife through butter.
Musicians and mortality. Here we consider what can go wrong as one goes from scholarly article to popular science piece to social media meme. We explore why a data graphic shared widely on social media gives a misleading impression, explain the issue of right-censoring, and discuss how its effects can be seen in the light of correlation analysis.
Traffic improvements. Irrelevant facts can lead you to bullshit conclusions if you approach them with an inaccurate model of how the world works. And if those conclusions let you spin a trite story about terrible traffic and wasteful government expenditures, what better clickbait?
99.9% caffeine-free. In one section of his book The Demon-Haunted World, Carl Sagan decried that way that advertisers try to dazzle us with irrelevant facts and figures. He was mostly concerned with drug advertising; in this case study we explore a more innocuous example.
A gender gap in 100 meter dash times. We examine a 2004 Nature paper predicting that women sprinters will outrun men by the mid-22nd century. In doing so, we see the danger of over-extrapolation, and we get to read a beautiful example of reductio ad absurdum as a means of calling bullshit.
Criminal machine learning. Machine learning algorithms are sometimes touted as generating results that unbiased and without prejudice. This is bullshit. We explore an example in which two authors claim to have an algorithm that can determine whether one is a criminal simply from an 80x80 facial image, and show that this algorithm is actually doing something very different.