Machine Learning · Chapter 7 of 26

Scaling, and the first transformers

By the end of this chapter you can say why a model that measures distance needs its columns on one scale, put them there with StandardScaler, MinMaxScaler and RobustScaler, read what each scaler learned, and keep the test rows out of what it learns. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.

  • Part: Preparing the data
  • 4 tasks
  • Python
  • In the all labs plan

The first 2 chapters of Machine Learning are free. This one comes with the all labs plan.

Why you need this

In chapter 2 Tanvi swapped the tree for a KNeighborsClassifier, kNN for short, as a black box. kNN guesses for a new applicant by finding the past loans that look most like them and seeing how those loans turned out. "Look like" means close, and close is measured as a distance between rows of numbers.

That is where Dobara's columns get in the way. monthly_income_rs is in rupees, tens of thousands of them; existing_emis runs from 0 to 5. A difference of ₹500 a month in income, which means almost nothing, counts 500 in the distance. A difference of three other EMIs, which Farida would call the whole story, counts 3. Given those numbers as they are, kNN looks for buyers who earn the same and barely notices anything else. On four of the loans' columns its test AUC is 0.55, hardly better than a coin. Nothing you have learned so far changes the units of a column, and picking new units by hand for every column is guessing again.

What you build: 4 tasks

Each task has tests. You write the code in the lab's editor and press Check.

  1. Task 1

    scale_train_test.py: scale the training rows, then the test rows

    Write scale_train_test(X_train, X_test) in scale_train_test.py. X_train and X_test are DataFrames of number columns, the two parts of a split: the same columns, different rows. Make a StandardScaler, fit it on X_train only, and transform both X_train and X_test with that one scaler.

    Return three values separated by commas: the fitted StandardScaler, the scaled training rows and the scaled test rows. Both tables are DataFrames that keep the columns' names and the rows' labels (set_output(transform="pandas") on the scaler does that); a NumPy array is not accepted.

    The test rows must be scaled with the mean and standard deviation learned from the training rows, never with their own. The Check tries the loans' five features of chapter 1 split with test_size=0.25 and random_state=7, three columns of the bikes, and 40 made-up loans, and it looks at what your scaler learned and from how many rows.

  2. Task 2

    knn_scaling.py: does scaling matter to kNN?

    Write knn_auc(X, y) in knn_scaling.py. X is a DataFrame of number columns and y the label (1 defaulted, 0 repaid). Split with train_test_split, test_size=0.25, random_state=7 and stratify=y.

    First fit a KNeighborsClassifier() with its default settings on the raw training rows, and find its ROC AUC on the raw test rows. Then fit a StandardScaler on the training rows only, fit a second KNeighborsClassifier() on the scaled training rows, and find its ROC AUC on the scaled test rows. Each AUC is roc_auc_score of the test labels and the probability of default (predict_proba's column for class 1), not of predict's 0s and 1s, and not the accuracy.

    Return two floats from 0 to 1, separated by a comma: the AUC on the raw columns first, then the AUC on the scaled columns. The Check tries the loans' five features of chapter 1, three other columns of the loans, and 400 made-up loans.

  3. Task 3

    km_scalers.py: seven typos and two scalers

    Seven bikes in bikes.csv have km_driven typed wrong, two of them as 999999. Write km_scaled(bikes) in km_scalers.py. bikes is a DataFrame with the columns of bikes.csv, or some of them (always bike_id and km_driven); do not change it.

    Fit a StandardScaler and a RobustScaler on the km_driven column of the bikes you are given, as a one-column table, and scale that column with each.

    Return a DataFrame with one row per bike, in the table's order and with its row labels, and exactly these columns in this order: bike_id, km_driven (as given), standard (the km scaled by the StandardScaler) and robust (the km scaled by the RobustScaler), both as floats. The Check tries all 1,600 bikes, the bikes without the seven typos, and 6 made-up bikes.

  4. Task 4 · interview question

    Interview: scaled_first.py, scaled before the split

    Raju scaled every loan with a MinMaxScaler, then split the scaled rows, fitted a kNN and reported its test AUC. Fix raju_knn(X, y) in scaled_first.py so that it splits X and y first with train_test_split, test_size=0.25, random_state=7 and stratify=y, fits the MinMaxScaler on the training rows only, fits a KNeighborsClassifier() with its default settings on the scaled training rows, and finds its ROC AUC on the scaled test rows (roc_auc_score with predict_proba's column for class 1).

    Return three values separated by commas: the fitted MinMaxScaler, the fitted KNeighborsClassifier, and the test AUC as a float from 0 to 1. The Check looks at what the scaler learned and from how many rows, and fits its own kNN on the same scaled training rows to compare. It tries the loans' five features of chapter 1 and 400 made-up loans.

    Tanvi will ask: Raju's AUC hardly changes when you fix it, so why is scaling before the split a leak, however small, and what did his scaler learn from the test rows?

What you will learn

  • Distance: how kNN finds the nearest loans, by hand with NumPy, and why one big column decides it.
  • Transformers: fit learns from the training rows, transform applies what it learned, fit_transform does both.
  • StandardScaler: mean_ and scale_, each column to mean 0 and standard deviation 1.
  • MinMaxScaler and RobustScaler: 0 to 1 from data_min_ and data_max_, or the median and the middle half from center_ and scale_, which a typo cannot move.
  • set_output: scaled tables that keep their column names.
  • The rule: fit the scaler on the training rows only.

More in Preparing the data

Build "Scaling, and the first transformers" yourself

Dobara refurbishes used bikes in Indore and lends to its buyers. You build the models that decide whom to lend to and at what price to sell. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.

The first 2 chapters of Machine Learning are free. This one comes with the all labs plan.