Course Lectures, Spring 2017

Calling Bullshit, Spring 2017

We will record and post all ten lectures from our Spring 2017 course. Each will be presented here as it becomes available. We have divided up every lecture into a set of a shorter segments; these segments should more or less stand alone on their own merits. The full playlist of all course videos is available on the UW Information School's YouTube channel.

Lecture 1: An Introduction to Bullshit

March 29, 2017

1.1 Introduction to Bullshit.
Bullshit is everywhere, and we've had enough. We want to teach people to detect and defuse bullshit where ever it may arise.

1.2 Calling Bullshit on Ourselves.
Jevin uses data graphics to boast about explosive growth at our website — and Carl calls bullshit. Old-school bullshit versus new-school bullshit.

1.3 Brandolini's Bullshit Asymmetry Principle.
Lecture 1.3 "The amount of effort necessary to refute bullshit is an order of magnitude bigger than to produce it."

1.4 Classroom Discussion.
Students discuss: What is bullshit anyway?

1.5 The Philosophy of Bullshit.
How do we define bullshit? Does intention matter? Calling bullshit as a speech act.

Lecture 2: Spotting Bullshit

April 5, 2017

2.1 Spotting Bullshit.
Jevin discusses some ways to spot bullshit and challenges students to tell whether four nuggets of wisdom from the internet are true or bullshit.

2.2 Sounds Too Good to be True.
If a claim seems too good — or too bad — to be true, it probably is. An example involving recommendation letters, and the perils of confirmation bias.

2.3 Entertain Multiple Hypotheses.
The importance of generating and considering multiple alternative hypotheses. As an example, we consider why men cite themselves more than women do.

2.4 Fermi Estimation.
Using Fermi estimation to check the plausibility of claims, with an example of food stamp fraud. This example is treated in further detail in one of our case studies.

2.5 Unfair Comparisons.
In this segment on unfair comparisons, Carl explains why St. Louis and Detroit are not quite as bad as clickbait "most dangerous cities" lists portray them to be, and looks at the silly arguments over attendance at Trump's inauguration. Also: how to call bullshit on algorithms and statistics without a PhD in machine learning or statistics.

2.6 Assignment: Bullshit Inventory.
In our first assignment, we ask students to take a week-long bullshit inventory of the bullshit they encounter, create, and debunk.

Lecture 3: Correlation and Causation

April 12, 2017

3.1 Correlation and Causation
Correlations are often used to make claims about causation. Be careful about the direction in which causality goes. For example: do food stamps cause poverty?

3.2 What are Correlations?
Jevin providers an informal introduction to linear correlations.

3.3 Spurious Correlations?
We look at Tyler Vigen’s silly examples of quantities appear to be correlated over time, and note that scientific studies may accidentally pick up on similarly meaningless relationships.

3.4 Correlation Exercise
When is correlation all you need, and causation is beside the point? Can you figure out which way causality goes for each of several correlations?

3.5 Common Causes
We explain how common causes can generate correlations between otherwise unrelated variables, and look at the correlational evidence that storks bring babies. We look at the need to think about multiple contributing causes. The fallacy of post hoc propter ergo hoc: the mistaken belief that if two events happen sequentially, the first must have caused the second.

3.6 Manipulative Experiments
We look at how manipulative experiments can be used to work out the direction of causation in correlated variables, and sum up the questions one should ask when presented with a correlation.

Lecture 4: Statistical Traps and Trickery

April 19, 2017

4.1 Right Censoring
We look at a graph of age at death for musicians in different genres, and use this to illustrate the problem of right-censored data. We consider this issue in further detail in one of our case studies.

4.2 Means and Medians
Simple as it may sound, the difference between mean and median values offers fertile ground for cooking up misleading statistics.

4.3 p-Values and the Prosecutor’s Fallacy
Carl presents what he thinks may be one of the most important segments in the whole course: a discussion of the prosecutor’s fallacy. This logical fallacy is not limited to the courtroom: it underlies a very common misinterpretation of the p values associated with scientific experiments.

4.4 The Will Rogers Effect
Will Rogers purportedly quipped that when the Okies left Oklahoma for California, they raised the average intelligence in both states. The same phenomenon can arise in epidemiology and a host of other areas.

4.5 Jevin's Turn
Jevin goes looking for bullshit and finds it — in Carl’s textbook. Jevin calls bullshit on Carl’s use of track and field records by age to illustrate senescence, and Carl tries to explain himself. This example is described further in another of our case studies.

Lecture 5: Big Data

April 26, 2017

5.1 Big Data Introduction
We briefly introduce big data and provide a few the cautionary tales surrounding this recent phenomenon. Beware of those ponies…

5.2 Garbage In, Garbage Out
You don’t need need a PhD in statistics or machine learning to call bullshit on big data. Simply by focusing on the input data and the results is often sufficient to refute a claim.

5.3 Big Data Hubris
We discuss the Google Flu Trends project and how it moved from being a poster child for big data to a providing an important precautionary tale.

5.4 Overfitting
We examine overfitting, the Achilles heel of machine learning. We illustrate overfitting visually, and consider and what to look out for.

5.5 Criminal Machine Learning
A recent paper claims that machine learning can determine whether or not you are a criminal from a photograph of your face. That's bullshit. This example is described further in one of our case studies.

5.6 Algorithmic Ethics
We discuss gender and racial biases inherent to many of the machine learning algorithms and recommender systems prevalent in today’s technology, and encourage others to call bullshit on machine injustice.