Machine Learning · Chapter 20 of 26
Reading what a model learned
By the end of this chapter you can say which columns a model leans on in three different ways, say when each one misleads (a column of random numbers, columns that share the credit), show how a model's risk moves as one column moves, and say plainly what none of it can tell you. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.
- Part: Tuning
- 4 tasks
- Python
- In the all labs plan
The first 2 chapters of Machine Learning are free. This one comes with the all labs plan.
Why you need this
Rachna is ready to lend by a model, and she has two questions before she does. "Which facts about a buyer matter most?" is the first: she must explain a refusal to a buyer, and to her bank. The second is harder: "If Dobara lowered EMIs, by lending for longer, would fewer buyers default?"
You have met one answer already, the tree's feature_importances_ (chapter 16), and logistic regression's coefficients (chapter 14). They come free with the fit, so it is tempting to read them as the truth. Tanvi will not, and this chapter shows why: add a column of pure random numbers to the loans and a forest's importances rank it fourth of eleven, above the bike's price. Each method answers a narrower question than it seems to, and the second question, about cause, is one no importance can answer.
What you build: 4 tasks
Each task has tests. You write the code in the lab's editor and press Check.
- Task 1
impurity.py: impurity importances
Write forest_importances(X, y) in impurity.py. X is a DataFrame of number columns (blanks allowed) and y the label defaulted. Split with train_test_split, test_size=0.25, random_state=7 and stratify=y. Fit a RandomForestClassifier with n_estimators=50, min_samples_leaf=5 and random_state=0 on the training rows only, and read its feature_importances_.
Return two values separated by a comma: the fitted forest, and a pandas Series of its importances labelled by X's column names, sorted from the highest to the lowest. The Check tries the loans' ten number columns and 600 made-up loans with other columns.
- Task 2
permutation.py: permutation importances
Write permutation_table(model, X_test, y_test) in permutation.py. model is a fitted classifier or pipeline, X_test a DataFrame of test rows it never saw, and y_test their labels. Call permutation_importance on the model and the test rows with scoring="roc_auc", n_repeats=5 and random_state=0; do not refit the model.
Return a DataFrame labelled by X_test's column names, with exactly the columns mean (the result's importances_mean) and std (its importances_std), sorted by mean from the highest to the lowest. The Check gives a logistic pipeline and a depth-3 tree with the loans' test rows, and a pipeline with 200 made-up test rows.
- Task 3
correlated.py: columns that share the credit
Write losses(X, y, group) in correlated.py. X is a DataFrame of number columns (blanks allowed), y the label defaulted, and group a list of some of X's column names. Split with train_test_split, test_size=0.25, random_state=7 and stratify=y. The model is always a fresh make_pipeline of SimpleImputer(strategy="median", add_indicator=True), StandardScaler() and LogisticRegression(max_iter=1000), fitted on training rows and scored by roc_auc_score of its predict_proba (second column) on the test rows. Fit it with every column; then without each column of the group in turn (the same column dropped from the training and the test rows); then without the whole group.
Return a dict with one key per column of the group and the key "together". Each value is the AUC lost: the AUC with every column minus the AUC without that column (for "together", without the whole group). The Check tries the loans with the group emi_rs, loan_rs and bike_price_rs, the loans with age and tenure_months, and made-up loans with two copies of one column.
- Task 4 · interview question
Interview: dependence.py, what if the EMI were lower?
Rachna asks whether fewer buyers would default if Dobara lowered their EMIs. Write average_risk(model, X, column, values) in dependence.py to show what the model says. model is a fitted classifier or pipeline, X a DataFrame of rows, column one of its column names, and values a list of numbers. For each value, in the list's order, take a copy of X with that column set to the value in every row, and the mean over the rows of the model's probability of default (the second column of predict_proba).
Return a pandas Series of those means, one per value, labelled by the values (partial dependence, by hand). Change a copy, never X itself. The Check gives a logistic pipeline and a forest with the loans' test rows, and a tree with made-up loans.
Tanvi will ask: the model's risk falls as the EMI falls. Does that show that lowering EMIs would cut defaults? What can the importances and this table say, and what not?
What you will learn
- Impurity importance (feature_importances_): free with any tree, and biased towards columns with many distinct values.
- Permutation importance (permutation_importance): shuffle one column of the test rows, see how far the score falls.
- Coefficients on scaled columns: a third answer, from a linear model.
- The trap all three share: correlated columns split the credit.
- What none of it can tell you: cause.
- Partial dependence (partial_dependence): the average predicted risk as one column moves, as a table.
More in Tuning
Build "Reading what a model learned" yourself
Dobara refurbishes used bikes in Indore and lends to its buyers. You build the models that decide whom to lend to and at what price to sell. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.
The first 2 chapters of Machine Learning are free. This one comes with the all labs plan.