Resources on improving user experiences (UX) for artificial intelligence (AI) with regard to ethics, diversity and inclusion are compiled here. These resources are primarily related to machine learning, but can be used for any technologies.
I’ll continue to add to this as I accumulate more.
Updated: December 11, 2020
Curiosity Activating UX Activities
Abusability Testing: UX in the Age of Abusability. The role of Composition, Collaboration, and Craft in building ethical products.
By Dan Brown. Sep 18, 2018.
Abusability Testing: Article describing the Abusability Testing workshop organized by Anna Abovyan, Theora Kvitka and Allison Cosby of the Pittsburgh IxDA Chapter for World Interaction Design Day 2019.
“Black Mirror” Episodes: Black Mirror, Light Mirror: Teaching Technology Ethics Through Speculation. By Casey Fiesler. Oct 15, 2018.
Implicit Association Test (IAT): via Harvard University.
Tools for Supporting Ethical AI
Checklist to prompt intentional, uncomfortable conversations: Designing Ethical AI Experiences: Checklist and Agreement by Carol Smith (me), Carnegie Mellon University, Software Engineering Institute.
Datasheets for Datasets
- Paper: “Datasheets for Datasets” on arxiv by Timnit Gebru, Jamie Morgenstern, Briana Vecchione, Jennifer Wortman Vaughan, Hanna Wallach, Hal Daumé III, Kate Crawford
- Markdown for Datasheets for Datasets by JRMeyer (GitHub — includes LaTeX version)
Ethics checklist for data scientists by Deon.
Lists of Additional Resources
Awesome AI Guidelines on GitHub from EthicalML
Mapping of ethical principles — Principled Artificial Intelligence: Mapping consensus in ethical and rights-based approaches to principles for AI by Jessica Fjeld and Adam Nagy, Harvard University — Download the visualization!
Organizations and Newsletters
Algorithm Watch newsletter
ML Engineer weekly newsletter from The Institute for Ethical AI & Machine Learning
Algorithms of Oppression: How Search Engines Reinforce Racism by Safiya Noble
Automating Inequality: How High-Tech Tools Profile, Police, and Punish the Poor Hardcover by Virginia Eubanks
Invisible Women: Data Bias in a World Designed for Men by Caroline Criado Perez
Rebooting AI: Building Artificial Intelligence We Can Trust by Gary Marcus, Ernest Davis
Technically Wrong: Sexist Apps, Biased Algorithms, and Other Threats of Toxic Tech by Sara Wachter-Boettcher
Weapons of Math Destruction by Cathy O’Neil
Carol Smith’s Contributions on AI Ethics and UX
My writings on the topic are below and you can also find my presentations on SlideShare.
“Designing Trustworthy AI for Human-Machine Teaming” — blog on the Carnegie Mellon University, Software Engineering Institute Blog which contains links to:
- Checklist to prompt intentional, uncomfortable conversations (same as linked above): Designing Ethical AI Experiences: Checklist and Agreement Carnegie Mellon University, Software Engineering Institute, Fact Sheet.
- “Designing Trustworthy AI: A Human-Machine Teaming Framework to Guide Development.” — Paper presented at AAAI (Association for the Advancement of Artificial Intelligence) Symposium FSS-19: Artificial Intelligence in Government and Public Sector. Hosted at Carnegie Mellon University.
“Creating a Curious, Ethical AI Workforce” — essay published in War on the Rocks.
“Intentionally Ethical AI Experiences” — Invited essay published in the Journal of Usability Studies by UXPA.