Science fiction and fantasy                                            

Weapons Of Math Destruction

by Cathy O'Neil

We are accustomed to seeing computers as neutral entities that don't hold grudges or harbour prejudices outside of the realm of science fiction. However, as we move into an age where algorithms rule most aspects of our lives, Cathy O'Neil takes a closer look at whether this trust is misplaced, and which kinds of algorithms constitute Weapons of Math Destruction (WMDs).

The author begins by relating an account of the 2007/8 financial crisis from her position within a hedge fund firm. She then proposes a number of factors that make the difference between a mathematical model that has a benign effect and one with the potential to cause great social, political, or economic harm.

Algorithms are all around us, and the way they are listed in the chapters of this book drives home how ubiquitous they have become and also, in some cases, how poor they are at predicting what they're meant to. A case study in college ranking scores delves into the social impact of each factor used to rate colleges, and also how these measures don't necessarily track what makes a college successful. For instance, a great college may have a low acceptance rate for students, but that percentage is only a proxy measure of the college's success rather than a result of it, and it could also be caused by other factors.

Education may have problems with mathematical models, including the distortions caused by college ranking systems, to teachers getting sacked as the result of flawed algorithms. Unfortunately, it's not the only area with difficulties. Online advertising profiling may deliver more relevant advertisements to some, but it also allows vulnerable people to be more easily targeted by scammers. Crime prediction models can end up unfairly targeting the poor, whilst systems perpetuate inequality in criminal justice. Getting a job, data-driven work scheduling, email monitoring, credit scores, insurance, health, and political gaming through social networks all come under the microscope. In each case shocking injustices emerge. There are situations where computer models are designed in ways that promote fairness, and the author highlights both good and bad uses to show how much difference a bit of forethought can make. Often, an algorithm can ruin someone financially or rob them of their freedom or health. The stakes are high.

Finally, the author makes a set of proposals for ways to keep Big Data in check. She also highlights a few cases where algorithms can be used in ways that could be extremely beneficial, and where a manual solution would be much less useful. This section on mitigating WMDs and finding positive aspects of them is brief, and it doesn't examine the politics behind companies' resistance to fixing poor algorithms. The issues involved may be far more complex than getting people to understand that there are problems with some algorithms and that there's resistance to dealing with those problems. However, the political aspect of algorithms is largely beyond the book's scope. Yet not acknowledging the scale of political inertia about the issues in this book makes the book's conclusions seem upbeat, even though it can take far more than a revelation of uncomfortable data to change policy. One of the author's anecdotes about the political reaction to some slides on homelessness backs up this point.

Weapons of Math Destruction is more focused on data science and how it can be improved, but the book makes clear throughout that data science can't be untangled from its political aspect. This is an essential and engaging read for anyone who wants to understand the impact mathematics has on our lives and how an objective model can be corrupted by accident or design to create systems that persecute groups or that are inherently unfair.

9th May 2018

Book Details

Year: 2016

Categories: Books


If you like this, try:

How To cover    

How To by Randall Munroe
In this book of popular science and maths, the author explores the answers to absurd questions about practical problems in the real world.

Physics Of The Future cover    

Physics Of The Future by Michio Kaku
An examination of how science could shape the world of 2100, and how it can help us to predict the near future.

5 star rating

Review ©

Source: own copy