Synopses & Reviews
An original account of willful ignorance and how this principle relates to modern probability and statistical methods Through a series of colorful stories about great thinkers and the problems they chose to solve, the author traces the historical evolution of probability and explains how statistical methods have helped to propel scientific research. However, the past success of statistics has depended on vast, deliberate simplifications amounting to willful ignorance, and this very success now threatens future advances in medicine, the social sciences, and other fields. Limitations of existing methods result in frequent reversals of scientific findings and recommendations, to the consternation of both scientists and the lay public.
Willful Ignorance: The Mismeasure of Uncertainty exposes the fallacy of regarding probability as the full measure of our uncertainty. The book explains how statistical methodology, though enormously productive and influential over the past century, is approaching a crisis. The deep and troubling divide between qualitative and quantitative modes of research, and between research and practice, are reflections of this underlying problem. The author outlines a path toward the re-engineering of data analysis to help close these gaps and accelerate scientific discovery.
Willful Ignorance: The Mismeasure of Uncertainty presents essential information and novel ideas that should be of interest to anyone concerned about the future of scientific research. The book is especially pertinent for professionals in statistics and related fields, including practicing and research clinicians, biomedical and social science researchers, business leaders, and policy-makers.
Synopsis
This provocative new book posits how the tacit principle of "willful ignorance" has led to a deep divide between qualitative and quantitative modes of research that will increasingly constrain scientific progress unless bridged by a broadened conception of statistical methodology. It presents numerous examples, both hypothetical and real, to illustrate and support the main premise; offers a non-technical, historical survey of core statistical concepts; and speculates about the future evolution of statistics. Clinicians will find this book to be of particular interest, as will others interested in observational statistics.
About the Author
Herbert I. Weisberg, PhD, is Founder of Causalytics, LLC, which develops innovative technology for predictive analytics for both medical research and business applications. He was previously President of Correlation Research Inc., a consulting firm specializing in the application of statistics to various business and legal issues. A Fellow of the American Statistical Association, Dr. Weisberg has published numerous articles and two previous books related to applied statistics.
Table of Contents
PREFACE xi
ACKNOWLEDGMENTS xv
1 THE OPPOSITE OF CERTAINTY 1Two Dead Ends 2
Analytical Engines 4
What is Probability? 6
Uncertainty 9
Willful Ignorance 12
Toward a New Science 15
2 A QUIET REVOLUTION 19Thinking the Unthinkable 21
Inventing Probability 24
Statistics 27
The Taming of Chance 31
The Ignorance Fallacy 34
The Dilemma of Science 35
3 A MATTER OF CHANCE 41Origins 43
Probability 44
The Famous Correspondence 56
What Did Not Happen Next 60
AgainstThe Odds 64
4 HARDLY TOUCHED UPON 71The Mathematics of Chance 73
Empirical Frequencies 82
A Quantum of Certainty 100
5 A MATHEMATICIAN OF BASEL 114Publication at Last 116
The Art of Conjecturing 117
A Tragic Ending 142
6 A DEFECT OF CHARACTER 147Man Without a Country 150
A Fraction of Chances 165
7 CLASSICAL PROBABILITY 171Revolutionary Reverends 173
From Chances to Probability 194
8 BABEL 213The Great Unraveling 216
Probability as a Relative Frequency 219
Probability as a Logical Relationship 228
Probability as a Subjective Assessment 239
Probability as a Propensity 247
9 PROBABILITY AND REALITY 253The Razor’s Edge 255
What Fisher Knew 257
What Reference Class? 262
A Postulate of Ignorance 270
Laplace’s Error 279
10 THE DECISION FACTORY 283Beyond Moral Certainty 284
Decisions, Decisions 298
Machine-Made Knowledge 309
11 THE LOTTERY IN SCIENCE 312Scientific Progress 313
Fooled by Causality 319
Statistics for Humans: Bias or Ambiguity? 331
Regression toward the Mean 339
12 TRUST, BUT VERIFY 346A New Problem 347
Trust,… 354
…But Verify 357
The Future 363
Mindful Ignorance 368
APPENDIX: THE PASCAL–FERMAT
CORRESPONDENCE OF 1654 373
NOTES 387
BIBLIOGRAPHY 415
INDEX 429