Cathy O’Neil

Mathematician for digital justice 

‘Algorithms are everywhere’, says Cathy O’Neil. Algorithms are increasingly governing our lives. Governments use them, financial institutions, schools and insurance companies. O’Neil: ‘They separate the winners from the losers.’

What are algorithms? Basically, they are sequences of instructions telling a computer what to do. Like when an online shop predicts what products you might buy based on your past purchases. Then the instruction is: ‘Customer buys product x. Look for products similar to x. Make a suggestion to customer.’ That seems like a pretty good service.

‘But what if the algorithms are wrong?’ asks American mathematician and data scientist Cathy O’Neil. Since the publication of her book Weapons of Math Destruction in 2016, O’Neil has become one of the most expressive voices warning against the injustices in the use of algorithms and Big Data. Coming from computers and being based on math, algorithms have the aura of neutrality. ‘But that’s a marketing trick’, O’Neil says in a Ted Talk on the subject. Algorithms are not objective truth, she emphasises. ‘They are opinions embedded in code.’

Of course, when an online shop suggests you buy a product you dislike, it’s not the end of the world. But algorithms can thoroughly affect people’s liberty, livelihood and finances. O’Neil: ‘A lot can go wrong if we put blind faith in Big Data.’ Weapons of Math Destruction is filled with examples of people whose lives were ruined after computer models took over from human judgment. Perfect teachers are fired, and well-functioning ex-detainees suddenly have to report every week. All based on the unknown criteria used by algorithms.

For outsiders, and even for users, it’s often impossible to track down how these models reach their conclusion. Do prejudices of the engineers play a role? Are the datasets flawed? ‘It’s all about who owns the code. The people who own the code, deploy it on other people’, O’Neil says in Coded Bias, the eye-opening documentary on artificial intelligence, racism and Big Data. ‘And it’s a totally asymmetrical power situation.’

For Cathy O’Neil, it’s crucial that the problem isn’t studied by just experts anymore. ‘We are cowed and overly trusting of algorithms because we are not experts in math and science’, she says in an interview. ‘I consider it much more of a political fight. It is a question about what is fair. And I don’t think anyone is like: sorry, I’m not qualified to talk about what is fairness.’

Coded Bias is shown at the Movies that Matter Festival 2021, where Cathy O’Neil will be a special guest.