Machine Learning · Chapter 8 of 26
Categorical features
By the end of this chapter you can turn text columns into numbers a model can learn from, with one column a category or with codes in a true order, send each column to the right encoder with a ColumnTransformer, and score new applications whose categories the model has never seen. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.
- Part: Preparing the data
- 4 tasks
- Python
- In the all labs plan
The first 2 chapters of Machine Learning are free. This one comes with the all labs plan.
Why you need this
Half of what Dobara knows about a buyer is words. Farmers defaulted at 25.4 % and salaried buyers at 8.2 %; a cruiser loan at 20.9 % and a commuter at 10.6 %. Farida reads the occupation on every application before she reads the income. Every model so far has learned from number columns only, because a model cannot learn from text. Give a tree the channel a buyer came through and it stops at once:
ValueError: could not convert string to float: 'dealer'
(the last line of the traceback, the error chapter 1 promised). You could replace each word with a number by hand, farmer 1, gig worker 2, and so on, but then the model reads "student is five times farmer", which means nothing, and every new word in next week's applications needs a new rule. This chapter turns words into numbers the way scikit-learn does it, so that the same encoding is used for training, testing and the week's applications.
What you build: 4 tasks
Each task has tests. You write the code in the lab's editor and press Check.
- Task 1
one_hot.py: occupation and segment, one column a category
Write one_hot(train, rows) in one_hot.py. Both are DataFrames with the columns of loans.csv as pd.read_csv reads it (rows may be applications instead): train holds the loans to learn from, rows the loans to encode. Make a OneHotEncoder with sparse_output=False, fit it on train's occupation and segment columns only (a table of those two columns, in that order), and transform the same two columns of rows.
Return two values separated by a comma: the fitted OneHotEncoder, and the encoded rows as a DataFrame whose columns are the encoder's own names (occupation_farmer, and so on, as get_feature_names_out gives them) and whose row labels are the rows' labels; a NumPy array is not accepted.
The encoder learns its categories from train, never from rows. The Check tries the training and test rows of a split of the loans, all the past loans with the new applications, and 6 made-up loans with 3 made-up applications that have only some of the categories.
- Task 2
condition_order.py: the bikes' condition, in its true order
Raju grades every bike poor, fair, good or excellent, from worst to best. Write condition_codes(bikes) in condition_order.py. bikes is a DataFrame with the columns of bikes.csv, or some of them (always condition); do not change it. Make an OrdinalEncoder whose categories are poor, fair, good and excellent in that order (one list, for the one column), fit it on the condition column as a one-column table, and transform that column.
Return two values separated by a comma: the fitted OrdinalEncoder, and the codes as a Series named condition_code with the bikes' row labels, in their order: 0.0 for poor, 1.0 for fair, 2.0 for good and 3.0 for excellent.
The Check tries all 1,600 bikes, only the good and excellent bikes (good is still 2.0 there), and 5 made-up bikes.
- Task 3
encode_columns.py: text encoded, numbers passed through
Write encode_columns(X_train, X_test) in encode_columns.py. Both are DataFrames with the same columns, some numbers and some text, the two parts of a split. Build one ColumnTransformer that one-hot encodes every column that is not a number with OneHotEncoder(sparse_output=False), the columns picked by their dtype with make_column_selector, passes every other column through as it is (remainder="passthrough"), names its output columns without a prefix (verbose_feature_names_out=False), and gives back a DataFrame (set_output(transform="pandas")). Fit it on X_train only, then transform X_test.
Return two values separated by a comma: the fitted ColumnTransformer, and the encoded test rows as a DataFrame: the one-hot columns first, then the number columns in their order, with the test rows' labels.
The Check gives tables with other columns: five columns of the loans, five of the bikes, and 8 made-up loans, so pick the text columns by dtype, never by name.
- Task 4 · interview question
Interview: new_dealers.py, this week's applications crash
Raju fitted a ColumnTransformer and a tree on all the past loans and scored this week's applications with them. It stops with ValueError: Found unknown categories ['D37', 'D38'] in column 1 during transform (the two dealers come in either order). Fix score_applications(loans, applications) in new_dealers.py so that Raju's OneHotEncoder for occupation and dealer has handle_unknown="ignore" and min_frequency=10 (and keeps sparse_output=False). The ColumnTransformer and the DecisionTreeClassifier(max_depth=4, random_state=0) are fitted on the past loans only, as Raju did, with the same columns.
Return three values separated by commas: the fitted ColumnTransformer, the fitted tree, and a NumPy array of each application's probability of default (predict_proba's column for class 1), one per application, in their order. The Check compares what your encoder learned with an encoder fitted on the past loans, and tries all the past loans and the first 1,200 of them, with the 250 new applications.
Tanvi will ask: what does the encoder do with D37 now, why group the dealers with fewer than 10 loans, and why must the encoder never be refitted on the new applications?
What you will learn
- OneHotEncoder: one 0-or-1 column a category, categories_, get_feature_names_out, sparse_output=False, drop=.
- Why pd.get_dummies is not for models: its columns depend on the rows it is given.
- OrdinalEncoder: one column of codes, in the order you give with categories=, and when an order is invented.
- ColumnTransformer: each column to its own transformer, remainder=, make_column_selector, verbose_feature_names_out.
- handle_unknown, min_frequency and max_categories: new and rare categories.
- TargetEncoder, in one section: a category as its default rate.
More in Preparing the data
Build "Categorical features" yourself
Dobara refurbishes used bikes in Indore and lends to its buyers. You build the models that decide whom to lend to and at what price to sell. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.
The first 2 chapters of Machine Learning are free. This one comes with the all labs plan.