Feel free to try the exercises below at your leisure. Solutions will
be posted later in the week!
Interpretive ML
Download the following test
set and model
(which was fit using caret). Using the iml
package, try to understand the model using the model-agnostic tools of
interpretive machine learning.
- Report basic diagnostics of the test set from the model. How well
does the model perform?
- Using permutation feature importance, identify the rank order of all
of the features to estimate the model. Which feature was the most
important?
- Present the individual conditional expectations with the partial
dependence plot overlaid across the range of values for the predictor
X.
- Estimate a global surrogate with the logit model and a decision tree
model. Present the regression table and the decision tree output. Which
model is a better surrogate?