Registro de citas para el Incidente 41
Entidades
Ver todas las entidadesEstadísticas de incidentes
Clasificaciones de la Taxonomía CSETv1
Detalles de la TaxonomíaHarm Distribution Basis
none
Sector of Deployment
professional, scientific and technical activities
Clasificaciones de la Taxonomía CSETv0
Detalles de la TaxonomíaFull Description
In 2018, MIT Media Lab researchers created an AI-powered "psychopath" text-generating algorithm named Norman. Norman was trained on caption data from a Reddit community that contained graphic images and videos about people dying. Following this training, they then showed Norman and a regular image recognition algorithm trained on the MSCOCO dataset a series of Rorschach inkblots, which psychologists have used to detect disorders. Norman's responses consistently described gruesome scenes, compared to innocent-sounding descriptions from the other algorithm; for example, "a black and white photo of a small bird," vs. "man gets pulled into dough machine." The researchers created Norman to demonstrate the influence training data has on how machine learning algorithms perform in the real world, and how poor data may lead to unreliable and untrustworthy outputs.
Short Description
MIT Media Lab researchers create AI-powered "psychopath" named Norman by training a model on "dark corners" of Reddit.
Severity
Negligible
Harm Type
Psychological harm
AI System Description
"Norman" is a text generating algorithm trained on disturbing content in order to demonstrate how training data can negatively affect an AI model. The comparison model is a regular text generation model.
System Developer
MIT Media Lab
Sector of Deployment
Information and communication
Relevant AI functions