Introduction to bullshit. What is bullshit? Concepts and categories of bullshit. The art, science, and moral imperative of calling bullshit. Brandolini's Bullshit Asymmetry Principle.
Spotting bullshit. Truth, like liberty, requires eternal vigilance. How do you spot bullshit in the wild? Effect sizes, dimensions, Fermi estimation, and checks on plausibility. Claims and the interests of those who make them. Forensic data analysis: GRIM test, Newcomb-Benford law.
The natural ecology of bullshit. Where do we find bullshit? Why do news media provide bullshit? TED talks and the marketplace for upscale bullshit. Why do social media provide ideal conditions for the growth and spread of bullshit?
Causality. One common source of bullshit data analysis arises when people ignore, deliberately or otherwise, the fact that correlation is not causation. The consequences can be hilarious, but this confusion can also be used to mislead. Confusing causality with necessity or sufficiency. Regression to the mean pitched as treatment effect. Milton Friedman's thermostat. Selection masked as transformation.
Statistical traps and trickery. Bayes rule and conditional probabilities. Base-rate fallacy/prosecutor's fallacy. Simpson's paradox. Data censoring. Will Rogers effect, lead-time bias, and length time bias. Means versus medians. Importance of higher moments.
Data visualization. Data graphics can be powerful tools for understanding information, but they can also be powerful tools for misleading audiences. We explore the many ways that data graphics can steer viewers toward misleading conclusions.
Big data. When does any old algorithm work given enough data, and when is it garbage in, garbage out? Use and abuse of machine learning. Misleading metrics. Goodhart's law.
Publication bias. Even a community of competent scientists all acting in good faith can generate a misleading scholarly record when ¿ as is the case in the current publishing environment ¿ journals prefer to publish positive results over negative ones. In a provocative and hugely influential 2005 paper, epidemiologist John Ioannides went so far as to argue that this publication bias has created a situation in which most published scientific results are probably false. As a result, it's not clear that one can safely rely on the results of some random study reported in the scientific literature, let alone on Buzzfeed. Once corporate funders with private agendas become involved, matters become all the more complicated.
Predatory publishing and scientific misconduct. Predatory publishing. The list formerly known as Beall's. Publishing economics. Pathologies of publish-or-perish culture. Pursuit of PR instead of progress. Data dredging, p-hacking, and similar malfeasance.
The ethics of calling bullshit. Where is the line between deserved criticism and targeted harassment? Is it, as one prominent scholar argued, ¿methodological terrorism¿ to call bullshit on a colleague's analysis? What if you use social media instead of a peer-reviewed journal to do so? How about calling bullshit on a whole field that you know almost nothing about? Pubpeer. Principles for the ethical calling of bullshit. The Dunning-Kruger effect.
Fake news. Fifteen years ago, nascent social media platforms offered the promise of a more democratic press through decentralized broadcasting and decoupling of publishing from advertising revenue. Instead, we get sectarian echo chambers and, lately, a serious assault on the very notion of fact.
Refuting bullshit. Refuting bullshit requires different approaches for different audiences.
Electronic Resources *