Arthur @ NeurIPS 2022
The Arthur team is thrilled to be attending NeurIPS again this year! We'll be in New Orleans, LA from Monday, November 28th through Friday, December 9th.
Keep reading to learn about the papers we'll be presenting and events we'll be hosting throughout the conference.
Let's Talk AI & Insurance!
Tensions Between the Proxies of Human Values in AI
Characterizing Anomalies with Explainable Classifiers
Teresa Datta, Daniel Nissani, Max Cembalest, Akash Khanna, Haley Massa, John P. Dickerson
Naveen Durvasula, Valentine d'Hauteville, Keegan Hines, John P. Dickerson
We explore how the current mathematical formalizations of fairness, privacy, and model transparency are imperfect, siloed constructions of the human values they hope to proxy, while giving the guise that those values are sufficiently embedded in our technologies. Tensions arise when practitioners attempt to achieve each pillar in isolation or simultaneously. In this paper, we push for redirection. By leaning on sociotechnical research, we can better understand how to evaluate a technology's embedded values.
- AFCP Workshop (Dec. 3, hybrid)
- HCAI Workshop (Dec. 9, virtual)
Model performance issues stemming from data drift can result in costly consequences. While methods exist to quantify data drift, a further classification of drifted points into groups of similarly anomalous points can be helpful for practitioners as a means to combating drift (e.g. by providing context about how/where in the data pipeline shift might be introduced). We show how such characterization is possible by making use of tools from the model explainability literature. We also show how simple rules can be extracted to generate database queries for anomalous data and detect it in the future.
- Workshop on Distribution Shifts: Connecting Methods and Applications (Dec. 3, in person)
Robustness Disparities in Face Detection
On the Generalizability and Predictability of Recommender Systems
Samuel Dooley, George Wei, Tom Goldstein, John P. Dickerson
Duncan McElfresh*, Sujay Khandagale*, Jonathan Valverde*, John P. Dickerson, Colin White
Facial analysis systems have been deployed by large companies and critiqued by scholars and activists for the past decade. We systematically investigate robustness disparities in face detection, an early stage in most facial analysis pipelines. We use both standard and recently released academic facial datasets to quantitatively analyze trends in face detection robustness. Across all the datasets and systems, we generally find that photos of individuals who are masculine presenting, older, of darker skin type, or have dim lighting are more susceptible to errors than their counterparts in other identities. We release code and data and hope this will spur additional analysis.
We create RecZilla, a meta-learning approach to recommender systems that uses a model to predict the best algorithm and hyperparameters for new, unseen datasets. By using far more meta-training data than prior work, RecZilla is able to substantially reduce the level of human involvement when faced with a new recommender system application. We not only release our code and pretrained RecZilla models, but also all of our raw experimental results, so that practitioners can train a RecZilla model for their desired performance metric.
Estimating Fairness in the Absence of Ground-Truth Labels
Michelle Bao, Jessica Dai, Keegan Hines, John P. Dickerson
- Women in Machine Learning Workshop: WiML 2022 (Nov. 28, hybrid)
LET'S GET TOGETHER
Joint with JPMC AI Research, Capital One, Bank of America, Schonfeld, Mila, IBM, Pfizer, Oxford, FINRA
The first half hour will be a reception, during which the organizers will pass around a sign-up sheet for topics and participants. Then, open mic debates and responses will commence. We’re seeking those folks in the AI/ML community with a primarily technical opinion that defies the status quo—and more importantly, a range of ideas, perspectives, and demographics.
Join us and our friends at Index Ventures on Wednesday, November 30 for drinks and bites at quintessential New Orleans venue Jack Rose.
Stay tuned for more info on how to RSVP!
MEET THE TEAM
Arthur's Machine Learning Team
Adam is the co-founder and CEO of Arthur, and has 20+ years of experience in the AI, ML, and software development spaces. Prior to founding Arthur, he founded and acted as CEO for Anax Security, a DC-based startup focused on ML for large-scale defensive cybersecurity. After Anax’s acquisition by Capital One, Adam served as Capital One’s VP of AI & Data Innovation. There, Adam helped bring AI observability, fairness, and explainability to high-value areas such as credit, UX, cybersecurity, and more.
Daniel is a researcher at Arthur interested in the ethical design and implementation of machine learning systems. Previously, he worked on synthetic data generation, specifically around unstructured text, at Gretel. He received a dual masters from Cornell Tech in Information Systems & Applied Information Sciences and a B.S. in Mathematics & Secondary Education from Northwestern University.
Hillary is Arthur's Chief Operations Officer. Prior to joining the Arthur team, she led skilled teams of engineers and strategists at Palantir Technologies, driving the company’s strategic European customer data infrastructure business and leading stability and regulatory initiatives for commercial partners. She holds a B.S. in Business Administration and Economics from Saint Louis University in Missouri.
John P. Dickerson
John is co-founder and Chief Scientist at Arthur, the AI performance monitoring company, as well as Associate Professor of Computer Science at the University of Maryland. His research centers on solving practical economic problems using techniques from computer science, stochastic optimization, and machine learning. He received his PhD in computer science from Carnegie Mellon University (SCS CSD PhD '16).
Karthik is a Machine Learning Engineer at Arthur. He was previously an undergraduate at Harvard focused on big data systems for machine learning. He has presented his work on counterfactual explanations using reinforcement learning at conferences such as ODSC East and Ray Summit. He is passionate about designing and building novel machine learning solutions using state-of-the art frameworks.
Keegan is the Vice President of Machine Learning at Arthur and an Adjunct Professor at Georgetown University. His PhD work was at the University of Texas in the lab of Rick Aldrich, with a focus on bringing powerful statistical and computational methods to bear on the study of protein biophysics. He is generally interested in how we can use machine learning in a reliable and trustworthy way.
Max is a researcher at Arthur focused on simplifying and explaining machine learning models. Previously, he received an M.S. in Data Science from Harvard University, where he concentrated on interpretability and graph-based models. He is particularly excited about recent advances in applying abstract algebra, topology, and category theory to neural network design.
Teresa is a researcher at Arthur interested in transparency and social impact of algorithmic systems from a human-centered lens. Previously, she worked on XAI and HCI projects while completing her M.S. in Data Science at Harvard University, and received her B.A. in Chemistry from Cornell University.
Valentine is a researcher at Arthur and is currently interested in data-centric approaches to improve model performance as well as algorithmic and design approaches to make AI broadly usable. She comes from a data science background. She recently completed a Computer Science master's degree at Columbia University and holds an undergraduate degree in Physics from the University of Pennsylvania.
Arthur is the AI Performance Company.
Our platform monitors, measures, and improves machine learning models to deliver better results. We help data scientists, product owners, and business leaders accelerate model operations and optimize for accuracy, explainability, and fairness.
With over 50+ years of combined industry and academic experience in AI and ML Operations, Arthur is the only company to adopt a research-led approach to product development. Our experts and experimental approach drive exclusive capabilities in computer vision, NLP, bias mitigation, and other critical areas.