About the project
In many applications, complex hard-to-interpret machine learning models like deep neural networks, random forests and gradient boosting machines are currently outperforming the traditional, and to some extent interpretable, linear/logistic regression models. However, often there is a clear trade-off between model complexity and model interpretability, meaning that it is often hard to understand why these sophisticated models perform so well. This lack of explanation constitutes a practical issue (“Can I trust the model?”), and a legal issue – those who develop the model can be required by law to explain what the model does to those who are exposed to automated decisions (the General Data Protection Regulation). In response, a new line of research has emerged that focuses on helping users to interpret the predictions from advanced machine learning methods.
Existing work on explaining complex models can be divided into two main categories; global and local explanations. The former tries to describe the model as a whole, in terms of which variables/ features that influenced the general model the most. Local explanations, on the other hand, try to identify how the different input variables/features influenced a specific prediction/output from the model, and are often referred to as individual prediction explanation methods.
Local explanation methods may further be divided into two categories: model-specific and model-agnostic (general) explanation methods. We are working mostly with methods in the latter category, with a special focus on Shapley-values and counterfactual explanations. The currently most used explanation methods based on Shapley values does not take feature dependence properly into account. This may lead to severely inaccurate estimates of feature contributions and incorrect explanations. Our main contribution is to properly handle dependence between variables both in the Shapley-based methods and counterfactual explanations.
Another problem with Shapley values is the computational complexity grows exponentially and becomes in-tractable for more than, say, ten features. There are approximations like e.g. the kernelSHAP method and the treeSHAP method, but these approaches have their weaknesses. Hence, in this project we are trying to develop better approximations.
Project team
- Senior research scientist Martin Jullum, Norwegian Computing Center
- Chief research scientist Anders Løland, Norwegian Computing Center
- Chief research scientist Kjersti Aas, Norwegian Computing Center
- Research scientist, Lars H. Olsen, Norwegian Computing Center