I originally published this article in Profiles in Diversity Journal. The article relates to my book, Predictive Analytics: The Power to Predict Who Will Click, Buy, Lie, or Die.
The Risk of Prejudice in Computerized Prediction
Today’s predictive technology introduces a new risk of prejudice on a massive, automated scale. Predictive analytics predicts what each person will do so that companies and government agencies can operate more effectively. In some cases, this technology does drive life-changing decisions. Large organizations like Hewlett-Packard inform human resource decisions by predicting whether each employee is likely to quit, and states such as Oregon and Pennsylvania analytically predict whether each convict will commit crime again (recidivism) in order to make sentencing and parole decisions.
Given this influence on the lives of individuals, predictive analytics introduces a new risk of prejudice in two ways:
1. Prediction of minority status. Fueled with data, computers automatically detect one’s minority status. A new study from the University of Cambridge shows that race, age, and sexual orientation can be accurately determined by one’s Facebook likes. The capacity to predict grants marketers and other researchers access to unvolunteered demographic information. Some such personnel may be keen on managing and using this information appropriately, but have not necessarily been trained to do so.
2. Prediction with minority status. When utilizing predictive analytics, it is difficult to avoid incorporating minority status into the predictive model as one basis of prediction. There is no place this threat is more apparent than in law enforcement, where computers have become respected advisers that have the attention of judges and parole boards.
While science promises to improve the effectiveness of law enforcement, when the organization formalizes and quantifies decision making, it inadvertently instills existing prejudices against minorities. Why? Because prejudice is cyclical, a self-fulfilling prophecy, and this cycling could be intensified by the deployment of predictive analytics.