data science · machine learning

Interpretability of ML

Global Interpretability

  • Partial Dependence and Partial Dependence Plot (PDP)
  • Individual Conditional Expectation (ICE)
  • Total and two-way H Statistics
  • Global Feature importance using permutation
  • Global surrogate model

Local Interpretability

  • Local Interpretable Model-agnostic Explanations (LIME)
  • Shapley additive explanation

An intuitive way to understand the Shapley value is the following illustration: The feature values enter a room in random order. All feature values in the room participate in the game (= contribute to the prediction). The Shapley value of a feature value is the average change in the prediction that the coalition already in the room receives when the feature value joins them.

The interpretation of the Shapley value is: Given the current set of feature values, the contribution of a feature value to the difference between the actual prediction and the mean prediction is the estimated Shapley value.


  • Independence: looks at the distribution of the scores.
  • Demographic Parity
  • Separation: looks at different performance metrics and ROC curves.
  • Equality of Opportunity, Equalized odds, Positive Predictive Parity,¬†Negative Predictive Parity, Accuracy Parity, Predictive Value Parity
  • Sufficiency: looks at the calibration curves (fraction of positive instances among set of all instances assigned a score value).


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s