An ongoing multistakeholder initiative to enable responsible AI by increasing transparency and accountability with machine learning system documentation.
ABOUT ML (Annotation and Benchmarking on Understanding and Transparency of Machine learning Lifecycles) is a multi-year, multi-stakeholder initiative led by PAI. This initiative aims to bring together a diverse range of perspectives to develop, test, and implement machine learning system documentation practices at scale.
The initiative is an ongoing, iterative process designed to co-evolve with the rapidly advancing field of AI development and deployment. In recognition that documentation is both an artifact and a process, ABOUT ML is structured into an artifact workstream and a process workstream. In 2020, ABOUT ML will produce one resource for each workstream:
A database of documentation questions, adapted by the domain of the machine learning application
A research-based guide to initiating and scaling a documentation pilot.
Documentation for machine learning systems can contribute to responsible AI development by bringing more transparency into “black box” models and by bridging the gap between increasingly pervasive AI ethics principles and day-to-day operations and practice. Documentation can shape practice because by asking the right question at the right time in the AI development process, teams will become more likely to identify potential issues and take appropriate mitigating actions.
Our goal for 2020 is to design testable pilots in a multi stakeholder manner. To make this process more tractable, we’ve broken this into two different workstreams. Each button below leads to a key subtask in this process, and we invite you to share your thoughts, comments, and feedback on any that you are interested in.
Help us solve the challenge of how documentation can be created at scale within an organization.
Comment on and Share your organizational challenges
Join the debate on what information stakeholders deserve to know about ML systems, and how that information should be presented.
Give feedback on deployed examples
Contribute to our database of questions
Implement and scale what works
Thanks for the submission!
The ABOUT ML Steering Committee is comprised of around 30 experts, researchers and practitioners recruited from a diverse set of PAI Partner organizations. The Steering Committee guides the process of updating ABOUT ML drafts based on the public comments submitted and new developments in research and practice. They vote to approve new releases by “rough consensus” commonly used by other multi-stakeholder working groups. They convene 1-3 times a year, depending on the volume of proposed changes and velocity of change of research and practice.
To allow for as many diverse perspectives as possible, PAI limits participation in the Steering Committee to up to 2 people per organization, with 1 vote per organization. As needed, PAI will periodically reopen applications to the Steering Committee and recruit more members.
Norberto Andrade
Privacy and Public Policy Manager (FACEbook)
Amir Banifatemi
General Manager, Innovation & Growth (XPRIZE)
Rachel Bellamy
Principal Researcher & Manager, Human-AI Collaboration (IBM)
Umang Bhatt
Student Fellow (Leverhulme Centre for the Future of Intelligence)
Rumman Chowdhury
Managing Director (Accenture AI)
Jacomo Corbo
Chief Scientist (QuantumBlack)
Daniel First
Associate / Data Scientist (McKinsey / QuantumBlack)
Ben Garfinkel
Research Fellow (Future of Humanity Institute)
Jeremy Gillula
Tech Projects Director (EFF)
Brenda Leong
Senior Counsel and Director of Strategy (Future of Privacy Forum)
Tyler Liechty
Data Engineer (DeepMind)
Momin M. Malik
Data Scientist (Berkman Klein Center)
Lassana Magassa
Graduate Research Associate (Tech Policy Lab/University of Washington)
Meg Mitchell
Researcher, ML Fairness, Ethical AI (Google)
Amanda Navarro
Managing Director (PolicyLink)
Deborah Raji
Tech Fellow (AI Now)
Nicole Rigillo
Anthropologist (Berggruen Institute)
Andrew Selbst
Postdoctoral Scholar (Data & Society)
Ramya Sethuraman
Product Manager (Facebook)
Moninder Singh
Research Staff Member (IBM)
Spandana Singh
Policy Analyst (Open Technology Institute)
Amber Sinha
Senior Programme Manager (Centre for Internet and Society)
Michael Spranger
Senior Research Scientist, AI Collaboration Office (Sony)
Andrew Strait
Researcher, Ethics and Society Team (DeepMind)
Gabriel Straub
Head of Data Science and Architecture (BBC)
Michael Veale
Assistant Professor (UCL)
Hanna Wallach
Senior Principal Researcher (Microsoft)
Adrian Weller
Senior Research Fellow (Leverhulme Centre for the Future of Intelligence)
Abigail Wen
Managing Counsel, Office of the CTO (Intel)
Alexander Wong
Co-Director, Vision and Image Processing (VIP) Research Group (University of Waterloo)
Jennifer Wortman Vaughan
Principal Researcher (Microsoft)
Andrew Zaldivar
Senior Developer Advocate (Google)
