The Alignment Problem Book Summary
Machine Learning and Human Values
Book by Brian Christian
Summary
The Alignment Problem explores the challenge of ensuring that as artificial intelligence systems grow more sophisticated, they reliably do what we want them to do - and argues that solving this "AI alignment problem" is crucial not only for beneficial AI, but for understanding intelligence and agency more broadly.
1. Prophecy
Bias in Machine Learning - Unrepresentative Training Data
Chapter 1 explores how bias and unfairness in machine learning models frequently stems from the data used to train them not being representative of the real world. Some key examples:
- Face recognition systems performing poorly on Black faces because their training data contained mostly White faces
- Word embedding models picking up on gender stereotypes because those associations were present in the large corpora of human-generated text used to train them
- Amazon's resume screening tool downranking women because it was trained on past resumes, which skewed male
The overarching lesson is that a model is only as unbiased as the data it learns from. Careful attention needs to be paid to the composition of training datasets to ensure they are adequately representative of the real-world populations the models will be applied to. There are also techniques to try to debias models, like identifying and removing stereotyped associations, but starting with representative data is the first line of defense against bias.
Section: 1, Chapter: 1
COMPAS Recidivism and Algorithmic Fairness
In 2016, a ProPublica investigation into the COMPAS criminal risk assessment tool concluded the tool was biased against Black defendants. Their analysis found that Black defendants who did not reoffend were 2x more likely to be classified as high-risk compared to White defendants.
The makers of COMPAS, Northpointe, countered that the model was equally accurate for White and Black defendants and had the same false positive rates for each risk score level, so could not be biased.
This sparked a heated debate in the algorithmic fairness community. A series of academic papers showed that the two notions of fairness - equal false positive rates and equal accuracy across groups - are mathematically incompatible if the base rates of the predicted variable differ across groups.
The COMPAS debate crystallized the realization that there are multiple conceptions of algorithmic fairness that often cannot be simultaneously satisfied. It brought the issue into the public eye and kickstarted the field of fairness in machine learning.
Section: 1, Chapter: 2
The 'Impossibility Of Fairness'
The COMPAS debate and subsequent academic work surface an unsettling truth: many desirable properties of machine learning models cannot be simultaneously satisfied. Specifically:
- Calibration: The model is equally accurate for all groups
- False positive equality: The model has equal false positive rates for all groups
- False negative equality: The model has equal false negative rates for all groups
Any two of these can be achieved, but satisfying all three is mathematically impossible if the base rates of the predicted variable (e.g. recidivism) differ between groups. Therefore:
- There is no 'perfect' definition of fairness in machine learning that is satisfying in all contexts
- We must make explicit value judgments about what properties are most important for a given application
- Striving to achieve all desirable properties in a single model is a fool's errand; we must prioritize
- We should be extremely cautious about over-relying on a single all-purpose model for high-stakes decisions
Section: 1, Chapter: 2
Machine Learning is not by Default Fair
“As we’re on the cusp of using machine learning for rendering basically all kinds of consequential decisions about human beings in domains such as education, employment, advertising, health care and policing, it is important to understand why machine learning is not, by default, fair or just in any meaningful way.”
Section: 1, Chapter: 2
The Simplest Models Are Often The Best
Chapter 3 makes the provocative case that often the most accurate models are the simplest ones, not complex neural networks, if the input features are wisely chosen.
Psychologist Paul Meehl showed in the 1950s that very simple statistical models consistently matched or beat expert human judgment at predicting things like academic performance or recidivism risk. Later work by Robyn Dawes in the 1970s demonstrated that even models with random feature weights (as long as they are positive) are highly competitive with human experts.
The key insight is that the predictive power comes from astute selection of the input features, not complex combinations of them. The experts' true skill is "knowing what to look for," then simple addition of those features does the rest.
This has major implications for model transparency. Wherever possible, simple, inspectable models should be preferred. And we should be extremely thoughtful about what features we choose to include since they, more than anything, drive the model's behavior.
Section: 1, Chapter: 3