A woman’s job application is rejected because of a recruiting algorithm that favors men’s résumés. A girl dies by suicide after graphic images of self-harm are pushed up on her feed by social media algorithms. A black teen steals something and gets rated high-risk for committing future crime by an algorithm used in courtroom sentencing, while a white man steals something of similar value and gets rated low-risk.
In recent years, advances in computer science have yielded algorithms so powerful that their creators have presented them as tools that can help us make decisions more efficiently and impartially. But the idea that algorithms are unbiased is a fantasy; in fact, they still end up reflecting human biases. And as they become ever more ubiquitous, we need to get clear on what they should — and should not — be allowed to do.
In a new book, A Human’s Guide to Machine Intelligence, Kartik Hosanagar, a University of Pennsylvania technology professor, argues we need an algorithmic bill of rights to protect us from the many risks AI is introducing into our lives, alongside the various benefits. Peoplehave called for such protections in the past, and in April, Sens. Cory Booker (D-NJ) and Ron Wyden (D-OR) introduced the Algorithmic Accountability Act. If passed, it would require companies to audit their algorithms for bias and discrimination. Some AI experts praised it as a “great first step” but noted that it leaves a number of concerns unaddressed.
All this got me wondering: Which demands, exactly, belong on an algorithmic bill of rights?
So I reached out to 10 experts (including Hosanagar) who are at the forefront of investigating how AI risk is creeping into the mundane aspects of life as well as high-stakes fields like immigration, medicine, and criminal justice. I asked them each to name a protection the public needs enshrined in law.
Allow me to present the result: a crowdsourced algorithmic bill of rights.
Transparency: We have the right to know when an algorithm is making a decision about us, which factors are being considered by the algorithm, and how those factors are being weighted.
Transparency is the No. 1 concern on people’s minds, judging by the responses I received. “We’re not even fully aware of when an algorithm is being used to make decisions for us or about us,” Hosanagar told me.
Say you’re applying for a mortgage. You deserve to know: Is an algorithm being used to make a decision about you? Is that decision based solely on the information you put down on your application form, or are your social media posts and other data obtained from third-party sources also being used? How does the mortgage approval algorithm rate different factors — does it place the greatest weight on income, medium weight on education, and low weight on current address, for example?
Cathy O’Neil, the author of Weapons of Math Destruction, put it this way in an email: “In situations where our financial lives, our livelihood, or our liberty is at risk — so not in the case of every algorithm under the sun — we should know what attributes about us are being used [and] we should know how our ‘scores’ depended on the values of those attributes.”
Explanation: We have the right to be given explanations about how algorithms affect us in a specific situation, and these explanations should be clear enough that the average person will be able to understand them.
Related to transparency is the demand for explainability. All algorithmic systems should carry something akin to a nutritional label laying out what went into them, according to Amy Webb, author of The Big Nine and founder of the Future Today Institute, which researches emerging technologies. “The terms of service for an AI application — or any service that uses algorithmic decision-making processes — should be written in language plain enough that a third grader can comprehend it,” she said. “It should be available in every language as soon as the application goes live.”
Consent: We have the right to give or refuse consent for any AI application that has a material impact on our lives or uses sensitive data, such as biometric data.
A demand for the right to consent has been gathering steam as more people realize that images of their faces are being used to power facial recognition technology. NBC reportedthat IBM had scraped a million photos of faces from the website Flickr — without the subjects’ or photographers’ permission. The news sparked a backlash. People may have consented to having their photos up on Flickr, but they hadn’t imagined their images would be used to train a technology that could one day be used to surveil them. Some states, like Oregon and Washington, are currently considering bills to regulate facial recognition.
The issue of consent extends well beyond that particular technology. Imagine you’re applying for a new job. Your prospective bosses inform you that your interview will be conducted by a robot — a practice that’s already in use today. Regardless of what they tout as the benefits of this AI system, you should have the right to give or withhold consent, according to MIT computer scientist Joy Buolamwini. “Permission must be granted,” she said, “not taken for granted.”
Freedom from bias: We have the right to evidence showing that algorithms have been tested for bias related to race, gender, and other protected characteristics — before they’re rolled out. The algorithms must meet standards of fairness and nondiscrimination and ensure just outcomes.
Like Buolamwini, who founded the Algorithmic Justice League to fight bias in automated decision-making systems, Yeshimabeit Milner is deeply concerned about AI that discriminates against people of color. As the founder and executive director of Data for Black Lives, Milner has drawn attention to problems with predictive policing (algorithmic systems for predicting where crime is likely to occur) and criminal risk assessments (algorithmic systems for predicting recidivism). Police officers and judges use both these systems to guide their decisions, despite evidence that they’re biased against black people.
Share our products, earn up to 40% per sale—apply today!