Fairness, transparency, and explainability are serious concerns faced by those building AI systems today.
Dataiku DSS empowers data scientists with the tools to build white-box machine learning models. One of these features designed to enable white-box machine learning is subpopulation analysis.
The Context for Subpopulation Analysis
For many use cases, the ability to explain or interpret the results of a machine learning model may be just as important as the accuracy of the predictions themselves.
A machine learning model is often labelled a black-box if its inner workings are largely obscured. A white-box model, on the other hand, allows for a more understandable relationship between input features and output predictions.
Subpopulation analysis is an example of a white-box ML technique.
Subpopulation Analysis with DSS
The nature of Dataiku DSS lends itself to white-box machine learning in a number of ways. The inherently collaborative nature of DSS and the ability to trace the lineage of data through a Flow are two fundamental properties that can increase understanding and trust in a model.
More specifically, DSS has features, such as partial dependence plots and subpopulation analysis, designed to make it easier to interpret model predictions.
You can compute a subpopulation analysis after building a regression or binary classification model trained in Python, such as scikit-learn or keras. The purpose of this analysis is to assess whether a model behaves identically across subpopulations.
This assessment is particularly important in a fairness study where you want to examine if a model reproduces biases in the training data. This tool can help users more easily weed out unintended model biases and create a more transparent and fair AI deployment.
This video below gives a brief introduction to the why and how of subpopulation analysis with DSS.
Interpreting Subpopulation Analysis
Subpopulation analysis computes a table of various statistics that you can compare across subpopulations that you define. For example, in a binary classification model, computing a subpopulation analysis on a variable such as gender allows you to examine the extent to which model prediction probabilities differ between male and female observations. In all cases, deciding whether a difference constitutes a significant finding with respect to fairness is left to you.
Does the model behave similarly for male and female subpopulations?
Consult the reference documentation for more information about how to conduct subpopulation analysis with Dataiku DSS.
The reference documentation also details how to create a partial dependence plot, which is another DSS tool for white-box machine learning.