The main idea of this book is to demonstrate a number of tricks used to misrepresent data in order to lead reader to preordained conclusions beneficial to data presenter. In short it is a nice guide to methods of creating “facts” and “proves” where there are none or spin real facts to such extend that they become irrelevant.
Introduction: Thinking, Critically
The brief introduction promotes the idea that users should not blindly trust any pseudo scientific ideas supported by graphs and numbers, but rather critically analyze what these data really show.
PART ONE: EVALUATING NUMBERS
The first step in this review is the test on plausibility. The sample of impossibility: something is down by 200%. Obviously nothing could be down by more than 100% at which point it becomes 0.
Fun with Averages
This is about typical misuse of Average vs. mean (same as average) vs. median (50% above/below) vs. mode (most frequent – pick of distribution). There is very nice example of company salary / bonus for owners income presentation manipulation to convey different points to different audiences.
This is about the use of graph with typical manipulation reviewed such as: Unlabeled Axes, Truncated vertical axis, Discontinuity of Axes, Improper scale of Axes, Double Y-axis with inadequate scales.
Hijinks with How Numbers Are Reported
This is about technics for misleading numerical data presentation such as: use of cumulative data to hide changes with nice example of cumulative sales data to hide drop in sales, plotting on the same graph unrelated data to imply correlation where there is none, use of deceptive illustrations and extrapolations with hilarious example of extrapolation decrease in coffee temperature to absolute 0, variation in data precision with nice example of manipulating birth data to prove point, and final trick – special subdividing with nice example of manipulation of statistics of heart disease death by subdividing them into categories in order to move another type of disease to the first place as cause of death.
How Numbers Are Collected
This is about manipulation of data collection in order to achieve preordained results such as: sampling biases, Participation Bias, Reporting Bias, Lack of standardization, Measurement error, and inclusion of unverifiable data.
This is a nice analysis of use of probabilities. It divides this use into classic probability based on symmetry, frequency probability based on known outcome, and subjective probability based on opinion (Bayesian probability). After that author briefly goes into the basics of statistics with probably most important point being that conditional probabilities are not invertible meaning that if probability of B is higher after A occurred, it does not mean that occurrence of B does not increase probability of A. Very good example number of car incidents at 7am vs. 7pm.
PART TINO: EVALUATING WORDS
How Do We Know?
This is a small chapter on veracity of quotations and expert opinions. The key phrase here is “It ain’t what you don’t know that gets you in trouble. It’s what you know for sure that just ain’t so”.
This is about identifying truthfulness of witnesses and real knowledge of experts. Author somewhat support trust into licensed experts, degreed experts, governmental experts, experts certified by prizes, authoritative publications and similar establishment supported validations. However he also provides some meaningful advice such as: True expertise is always narrow, Check website domains and links to find out who is really behind catchy name, Check for time relevance of websites, Take into account institutional bias.
Overlooked, Undervalued Alternative Explanations
When evaluation claims, take into account alternative from suggested reasons for factual events. Exclude non-provable and therefor meaningless explanations: like “Aliens did it”. For research result check existence and validity of control group, sample sizes, selection criteria, and overall statistical validity of claims.
This is about misinformation packaged to look like a fact. Author provides examples of step-by-step building a misinformation case based on accumulation small plausible, but not necessarily correct statements. One of very popular misleading technics providing facts without link to big picture something like “double number of hurricanes in area A during last year means huge increase in X” without mentioning that previous year there were 1/10 of average number of last 100 years, meaning that X is not changing or maybe even decreasing. Another very nice technic of persuasion is to provide a number of weakly related correct facts in congestion with false statement one wants to promote. Here is a nice example for selling bottled water:
PART THREE: EVALUATING THE WORLD
How Science Works
This is a nice small presentation of formal methods of deduction and induction.
Here author discusses sample limitation leading to illusory correlation, framing probabilities and risks, and believe perseverance when contradictory evidence is heavily discounted. As example author provide story about link between autism and vaccines.
Knowing What You Don’t Know
This is a brief discussion of 2X2 Known / unknown matrix popularized by Rumsfeld with stress on need to minimize unknown/unknown, which is the most dangerous form of ignorance.
Bayesian Thinking in Science and in Court Four Case Studies
This is another brief discussion this time on Bayesian thinking: conditional probabilities.
Four Case Studies
Here author provides 4 case studies to demonstrate difficulties of decision-making and supporting information selection in contemporary world overwhelmed with various data often presented in such way that it is very difficult to obtain valid representation of reality. The cases are:
- Author dog’s fatal illness and related decision making process
- Reality of moon landing
- David Blaine’s presentations of non-breathing or ice cube frizzing
- Pattern recognition in in periodic table and use of statistics in Standard Model of Particle Physics
Conclusion: Discovering Your Own
The conclusion is the call to use one’s own intellectual facilities to recognize misleading information and data presentation because the world is filled with agenda currying presentations and lack of vigilance would lead to costly mistakes in decision making.
MY TAKE ON IT:
In my opinion it is a very valuable book, even if for me personally there is very little new in this presentation. Having spent first 37 years of my live in Soviet Union, that really had Orwellian ministries of truth and propaganda was considered a necessary tool of mass education, I learned to read between lines, decode graphs and tables to identify truthful information that these graphs and tables where designed to hide. However such training was not available for everybody so lots of people in western world get lost without such skills and often buy into all kinds of misrepresentations causing them significant harm in many areas from dietary behavior negatively impacting their health to political behavior negatively impacting their material well-being. Paradoxically author of this book on more than one occasion demonstrate his leftist leaning, clearly demonstrating how it impacts his judgment in presenting examples of misleading.