I first heard of Weapons of Math Destruction in a radio interview with the author. She was quite engaging, but how would a book on mathematical modelling stack up?
A former Wall Street quant sounds an alarm on mathematical modeling—a pervasive new force in society that threatens to undermine democracy and widen inequality.
We live in the age of the algorithm. Increasingly, the decisions that affect our lives—where we go to school, whether we get a car loan, how much we pay for health insurance—are being made not by humans, but by mathematical models. In theory, this should lead to greater fairness: Everyone is judged according to the same rules, and bias is eliminated. But as Cathy O’Neil reveals in this shocking book, the opposite is true. The models being used today are opaque, unregulated, and uncontestable, even when they’re wrong. Most troubling, they reinforce discrimination: If a poor student can’t get a loan because a lending model deems him too risky (by virtue of his race or neighborhood), he’s then cut off from the kind of education that could pull him out of poverty, and a vicious spiral ensues. Models are propping up the lucky and punishing the downtrodden, creating a “toxic cocktail for democracy.” Welcome to the dark side of Big Data.
Tracing the arc of a person’s life, from college to retirement, O’Neil exposes the black box models that shape our future, both as individuals and as a society. Models that score teachers and students, sort resumes, grant (or deny) loans, evaluate workers, target voters, set parole, and monitor our health—all have pernicious feedback loops. They don’t simply describe reality, as proponents claim, they change reality, by expanding or limiting the opportunities people have. O’Neil calls on modelers to take more responsibility for how their algorithms are being used. But in the end, it’s up to us to become more savvy about the models that govern our lives. This important book empowers us to ask the tough questions, uncover the truth, and demand change.
Why I Love It:
Weapons of Math Destruction is a fascinating – and terrifying – read. I love titles that work on many levels and this one is brilliant. Brilliant pun and yet incredibly relevant once you start reading about the damage these mathematic models are doing to society in all areas from the justice system to education to consumer profiling.
These models, used in all walks of life, supposedly to streamline or enhance an area are built with inherent biases, resulting in incredible damage to many people, often the most marginalized among us. And because there is no transparency in how these models are determined or even, in many cases, an awareness that people are being modelled, there are no opportunities to push back. Even those that do demand accountability tend to be dismissed because, well, the model has spoken, and is taken as the word of God.
I had no idea how widely or how perniciously these models are being employed. This is a very important book that I think all us of should be reading. I appreciate how O’Neil lays these concepts out in an easy-to-follow manner, with plenty of real-world examples. Plus, she gets into the ethics of this kind of data mining and possible ways it could be used for good. It makes for an engaging read. I don’t read non-fiction all that often but this was one of those books like Freakonomics or The Brain That Changes Itself that will stay with me for a long time.