Sidestepping the Black-Box: A New Paradigm for Explainable AI

Existing work in Explainable Artificial Intelligence (XAI) has been focused on developing techniques to interpret decisions made by pre-trained & black-box machine learning (ML) models. This black-box assumption is reasonable in a lot of settings, e.g., explaining Amazon’s recommender systems requires assuming a black-box model because it is infeasible to assume glass-box access to Amazon’s proprietary models, etc. However, I argue that in many real-world settings (especially those that pertain to low-resource domains), the black-box assumption is unnecessary, undesirable, and often, overly limiting. In this talk, I motivate the need to move away from the black-box assumption of XAI by discussing two deployed use cases of responsible AI research: (i) automated tele-triage for poor pregnant women in Kenya, and (ii) raising awareness of HIV among homeless youth in Los Angeles. Through my experiences with the deployment of AI in these domains, we will argue the need for a new paradigm in explainable AI. Next, I will discuss two new frameworks: (i) CounterNet, a novel end-to-end learning framework which integrates Machine Learning (ML) model training and the generation of corresponding counterfactual (CF) explanations into a single end-to-end pipeline; and (ii) RoCourseNet, a training framework that jointly optimizes predictions and recourses that are robust to future data shifts.

Amulya Yadav is the PNC Technologies Career Development Assistant Professor in the College of Information Sciences and Technology at Penn State University, where he serves as Director of the RAISE Research Lab. He is also the Associate Director (Programs) at the Center for Socially Responsible AI @ Penn State. Amulya’s research work in the field of Responsible AI and Artificial Intelligence for Social Good focuses on developing theoretically grounded approaches to real-world problems that can have an impact in the field. His algorithms have been deployed in the real-world, particularly in the field of public health and wildlife protection. Amulya is a recipient of the AAMAS 2016 Best Student Paper Award, the AAAI 2017 Best Video and Best Student Video Award, the IDEAS 2016 Most Visionary Paper Award, and the AAMAS 2017 Best Paper Award nomination. His work has also been highlighted by Mashable.com as one of 26 incredible innovations that improved the world in 2015.

Amulya holds a Ph.D. in Computer Science from the University of Southern California, and a B. Tech. in Computer Science & Engineering from Indian Institute of Technology (IIT), Patna.