Product      Blog      Our Team      Careers

The Arthur team is thrilled to be attending NeurIPS again this year! We'll be in New Orleans, LA from Monday, November 28th through Friday, December 9th. 

Keep reading to learn about the papers we'll be presenting and events we'll be hosting throughout the conference.

Tensions Between the Proxies of Human Values in AI

Characterizing Anomalies with Explainable Classifiers

Teresa Datta, Daniel Nissani, Max Cembalest, Akash Khanna, Haley Massa, John P. Dickerson

Naveen Durvasula, Valentine d'Hauteville, Keegan Hines, John P. Dickerson

We explore how the current mathematical formalizations of fairness, privacy, and model transparency are imperfect, siloed constructions of the human values they hope to proxy, while giving the guise that those values are sufficiently embedded in our technologies. Tensions arise when practitioners attempt to achieve each pillar in isolation or simultaneously. In this paper, we push for redirection. By leaning on sociotechnical research, we can better understand how to evaluate a technology's embedded values. 

Workshops:

- AFCP Workshop (Dec. 3, hybrid)

- HCAI Workshop (Dec. 9, virtual)

Model performance issues stemming from data drift can result in costly consequences. While methods exist to quantify data drift, a further classification of drifted points into groups of similarly anomalous points can be helpful for practitioners as a means to combating drift (e.g. by providing context about how/where in the data pipeline shift might be introduced). We show how such characterization is possible by making use of tools from the model explainability literature. We also show how simple rules can be extracted to generate database queries for anomalous data and detect it in the future.

Workshop:

- Workshop on Distribution Shifts: Connecting Methods and Applications (Dec. 3, in person)

Robustness Disparities in Face Detection

On the Generalizability and Predictability of Recommender Systems

Samuel Dooley, George Wei, Tom Goldstein, John P. Dickerson

Duncan McElfresh*, Sujay Khandagale*, Jonathan Valverde*, John P. Dickerson, Colin White

Facial analysis systems have been deployed by large companies and critiqued by scholars and activists for the past decade. We systematically investigate robustness disparities in face detection, an early stage in most facial analysis pipelines. We use both standard and recently released academic facial datasets to quantitatively analyze trends in face detection robustness. Across all the datasets and systems, we generally find that photos of individuals who are masculine presenting, older, of darker skin type, or have dim lighting are more susceptible to errors than their counterparts in other identities. We release code and data and hope this will spur additional analysis.

We create RecZilla, a meta-learning approach to recommender systems that uses a model to predict the best algorithm and hyperparameters for new, unseen datasets. By using far more meta-training data than prior work, RecZilla is able to substantially reduce the level of human involvement when faced with a new recommender system application. We not only release our code and pretrained RecZilla models, but also all of our raw experimental results, so that practitioners can train a RecZilla model for their desired performance metric.

Links:

- Read the paper

- See the code

Estimating Fairness in the Absence of Ground-Truth Labels

Michelle Bao, Jessica Dai, Keegan Hines, John P. Dickerson

Workshop:

- Women in Machine Learning Workshop: WiML 2022 (Nov. 28, hybrid)

Joint with JPMC AI Research, Capital One, Bank of America, Schonfeld, Mila, IBM, Pfizer, Oxford, FINRA

The first half hour will be a reception, during which the organizers will pass around a sign-up sheet for topics and participants. Then, open mic debates and responses will commence. We’re seeking those folks in the AI/ML community with a primarily technical opinion that defies the status quo—and more importantly, a range of ideas, perspectives, and demographics.

Join us and our friends at Index Ventures on Wednesday, November 30 for drinks and bites at quintessential New Orleans venue Jack Rose.

Stay tuned for more info on how to RSVP!

Adam Wenchel

Adam is the co-founder and CEO of Arthur, and has 20+ years of experience in the AI, ML, and software development spaces. Prior to founding Arthur, he founded and acted as CEO for Anax Security, a DC-based startup focused on ML for large-scale defensive cybersecurity. After Anax’s acquisition by Capital One, Adam served as Capital One’s VP of AI & Data Innovation. There, Adam helped bring AI observability, fairness, and explainability to high-value areas such as credit, UX, cybersecurity, and more.

Daniel Nissani

Daniel is a researcher at Arthur interested in the ethical design and implementation of machine learning systems. Previously, he worked on synthetic data generation, specifically around unstructured text, at Gretel. He received a dual masters from Cornell Tech in Information Systems & Applied Information Sciences and a B.S. in Mathematics & Secondary Education from Northwestern University.

Hillary Clark

Hillary is Arthur's Chief Operations Officer. Prior to joining the Arthur team, she led skilled teams of engineers and strategists at Palantir Technologies, driving the company’s strategic European customer data infrastructure business and leading stability and regulatory initiatives for commercial partners. She holds a B.S. in Business Administration and Economics from Saint Louis University in Missouri. 

John P. Dickerson

John is co-founder and Chief Scientist at Arthur, the AI performance monitoring company, as well as Associate Professor of Computer Science at the University of Maryland. His research centers on solving practical economic problems using techniques from computer science, stochastic optimization, and machine learning. He received his PhD in computer science from Carnegie Mellon University (SCS CSD PhD '16).

Karthik Rao

Karthik is a Machine Learning Engineer at Arthur. He was previously an undergraduate at Harvard focused on big data systems for machine learning. He has presented his work on counterfactual explanations using reinforcement learning at conferences such as ODSC East and Ray Summit. He is passionate about designing and building novel machine learning solutions using state-of-the art frameworks.

Keegan Hines

Keegan is the Vice President of Machine Learning at Arthur and an Adjunct Professor at Georgetown University. His PhD work was at the University of Texas in the lab of Rick Aldrich, with a focus on bringing powerful statistical and computational methods to bear on the study of protein biophysics. He is generally interested in how we can use machine learning in a reliable and trustworthy way.

Max Cembalest

Max is a researcher at Arthur focused on simplifying and explaining machine learning models. Previously, he received an M.S. in Data Science from Harvard University, where he concentrated on interpretability and graph-based models. He is particularly excited about recent advances in applying abstract algebra, topology, and category theory to neural network design.

Teresa Datta

Teresa is a researcher at Arthur interested in transparency and social impact of algorithmic systems from a human-centered lens. Previously, she worked on XAI and HCI projects while completing her M.S. in Data Science at Harvard University, and received her B.A. in Chemistry from Cornell University.

Valentine d'Hauteville

Valentine is a researcher at Arthur and is currently interested in data-centric approaches to improve model performance as well as algorithmic and design approaches to make AI broadly usable. She comes from a data science background. She recently completed a Computer Science master's degree at Columbia University and holds an undergraduate degree in Physics from the University of Pennsylvania.

Arthur is the AI Performance Company.

Our platform monitors, measures, and improves machine learning models to deliver better results. We help data scientists, product owners, and business leaders accelerate model operations and optimize for accuracy, explainability, and fairness.

With over 50+ years of combined industry and academic experience in AI and ML Operations, Arthur is the only company to adopt a research-led approach to product development. Our experts and experimental approach drive exclusive capabilities in computer vision, NLP, bias mitigation, and other critical areas.

LinkedIn     Twitter      Our Team      Careers