Incident 89: The Christchurch shooter and YouTube’s radicalization trap

Description: A New Zealand government report released following a right-wing terrorist killing 51 worshippers at two New Sealand mosques which indicated that Youtube's recommendation algorithm played an important role in the terrorist's radicalization.

Tools

New Report New Response DiscoverView History

Entities

View all entities

Alleged: YouTube developed and deployed an AI system, which harmed YouTube users.

Incident Stats

Incident ID

Report Count

Incident Date

2019-03-15

Editors

Sean McGregor, Khoa Lam

CSETv0 Taxonomy Classifications

Taxonomy Details

Full Description

A New Zealand government report released following a right-wing terrorist killing 51 worshippers at two New Zealand mosques which indicated that Youtube's recommendation algorithm played an significant role in the terrorist's radicalization.The New Zealand government argues that social media platforms like YouTube claim that these algorithms should be publicly available for accountability, and that simple removal and identification is not enough because of cases like the radicalization of the terrorist.

Short Description

A New Zealand government report released following a right-wing terrorist killing 51 worshippers at two New Sealand mosques which indicated that Youtube's recommendation algorithm played an important role in the terrorist's radicalization.

Severity

Moderate

Harm Distribution Basis

Race, Religion

Harm Type

Psychological harm

AI System Description

The YouTube recommendation algorithm recommends videos to users based on previous user interactions with the platform. It ranks videos by assigning them a score based on performance analytics data and it matches videos to people based on their watch history, and what similar people have watched.

System Developer

Google

Sector of Deployment

Information and communication

Relevant AI functions

Cognition, Action

AI Techniques

machine learning

AI Applications

content learning, recommendation engine, forecasting

Location

New Zealand

Named Entities

New Zealand, YouTube

Technology Purveyor

Google

Beginning Date

03/2019

Ending Date

03/2019

Near Miss

Harm caused

Intent

Accident

Lives Lost

Yes

Data Inputs

user history, videos

Incident Reports

Reports Timeline

The Christchurch shooter and YouTube’s radicalization trap

wired.com

wired.com · 2020

YOUTUBE, FACEBOOK, AND other social media platforms were instrumental in radicalizing the terrorist who killed 51 worshippers in a March 2019 attack on two New Zealand mosques, according to a new report from the country’s government. Online…

Variants

A "variant" is an incident that shares the same causative factors, produces similar harms, and involves the same intelligent systems as a known AI incident. Rather than index variants as entirely separate incidents, we list variations of incidents under the first similar incident submitted to the database. Unlike other submission types to the incident database, variants are not required to have reporting in evidence external to the Incident Database. Learn more from the research paper.