1.1 Introduction to Bullshit.
Bullshit is everywhere, and we've had enough. We want to teach people to detect and defuse bullshit where ever it may arise.
1.2 Calling Bullshit on Ourselves.
Jevin uses data graphics to boast about explosive growth at our website callingbullshit.org — and Carl calls bullshit. Old-school bullshit versus new-school bullshit.
1.3 Brandolini's Bullshit Asymmetry Principle.
Lecture 1.3 "The amount of effort necessary to refute bullshit is an order of magnitude bigger than to produce it."
1.4 Classroom Discussion.
Students discuss: What is bullshit anyway?
1.5 The Philosophy of Bullshit.
How do we define bullshit? Does intention matter? Calling bullshit as a speech act.
2.1 Spotting Bullshit.
Jevin discusses some ways to spot bullshit and challenges students to tell whether four nuggets of wisdom from the internet are true or bullshit.
2.2 Sounds Too Good to be True.
If a claim seems too good — or too bad — to be true, it probably is. An example involving recommendation letters, and the perils of confirmation bias.
2.3 Entertain Multiple Hypotheses.
The importance of generating and considering multiple alternative hypotheses. As an example, we consider why men cite themselves more than women do.
2.5 Unfair Comparisons.
In this segment on unfair comparisons, Carl explains why St. Louis and Detroit are not quite as bad as clickbait "most dangerous cities" lists portray them to be, and looks at the silly arguments over attendance at Trump's inauguration. Also: how to call bullshit on algorithms and statistics without a PhD in machine learning or statistics.
2.6 Assignment: Bullshit Inventory.
In our first assignment, we ask students to take a week-long bullshit inventory of the bullshit they encounter, create, and debunk.
3.1 Correlation and Causation
Correlations are often used to make claims about causation. Be careful about the direction in which causality goes. For example: do food stamps cause poverty?
3.2 What are Correlations?
Jevin providers an informal introduction to linear correlations.
3.3 Spurious Correlations?
We look at Tyler Vigen’s silly examples of quantities appear to be correlated over time, and note that scientific studies may accidentally pick up on similarly meaningless relationships.
3.4 Correlation Exercise
When is correlation all you need, and causation is beside the point? Can you figure out which way causality goes for each of several correlations?
3.5 Common Causes
We explain how common causes can generate correlations between otherwise unrelated variables, and look at the correlational evidence that storks bring babies. We look at the need to think about multiple contributing causes. The fallacy of post hoc propter ergo hoc: the mistaken belief that if two events happen sequentially, the first must have caused the second.
3.6 Manipulative Experiments
We look at how manipulative experiments can be used to work out the direction of causation in correlated variables, and sum up the questions one should ask when presented with a correlation.
4.1 Right Censoring
We look at a graph of age at death for musicians in different genres, and use this to illustrate the problem of right-censored data. We consider this issue in further detail in one of our case studies.
4.2 Means and Medians
Simple as it may sound, the difference between mean and median values offers fertile ground for cooking up misleading statistics.
4.3 p-Values and the Prosecutor’s Fallacy
Carl presents what he thinks may be one of the most important segments in the whole course: a discussion of the prosecutor’s fallacy. This logical fallacy is not limited to the courtroom: it underlies a very common misinterpretation of the p values associated with scientific experiments.
4.4 The Will Rogers Effect
Will Rogers purportedly quipped that when the Okies left Oklahoma for California, they raised the average intelligence in both states. The same phenomenon can arise in epidemiology and a host of other areas.
4.5 Jevin's Turn
Jevin goes looking for bullshit and finds it — in Carl’s textbook. Jevin calls bullshit on Carl’s use of track and field records by age to illustrate senescence, and Carl tries to explain himself. This example is described further in another of our case studies.
5.1 Big Data Introduction
We briefly introduce big data and provide a few the cautionary tales surrounding this recent phenomenon. Beware of those ponies…
5.2 Garbage In, Garbage Out
You don’t need need a PhD in statistics or machine learning to call bullshit on big data. Simply by focusing on the input data and the results is often sufficient to refute a claim.
5.3 Big Data Hubris
We discuss the Google Flu Trends project and how it moved from being a poster child for big data to a providing an important precautionary tale.
We examine overfitting, the Achilles heel of machine learning. We illustrate overfitting visually, and consider and what to look out for.
5.5 Criminal Machine Learning
A recent paper claims that machine learning can determine whether or not you are a criminal from a photograph of your face. That's bullshit. This example is described further in one of our case studies.
5.6 Algorithmic Ethics
We discuss gender and racial biases inherent to many of the machine learning algorithms and recommender systems prevalent in today’s technology, and encourage others to call bullshit on machine injustice.