Machine Learning · Chapter 25 of 26

Capstone: Raju's price model, end to end

By the end of this chapter you have built Dobara's price model from the raw bikes to a number Raju trusts: a fair split, the floor, a pipeline that handles an ordered grade, a missing engine and mistyped kilometres, a log target that turns a curve into a line, a small search, a diagnosis of where it misses, and a report in rupees and in "how often is it close enough". Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.

  • Part: Real work
  • 4 tasks
  • Python
  • In the all labs plan

The first 2 chapters of Machine Learning are free. This one comes with the all labs plan.

Why you need this

Raju prices every bike Dobara buys. Pay a seller too much and the margin is gone; ask a buyer too much and the bike sits in the showroom. He prices from experience, and he is good, but Dobara now buys more bikes than one person can look at, across six branches. Rachna wants a model that gives every bike a fair price on the day it is listed, and Farida wants the same number for her loans: a loan is safer when it is small next to what the bike is worth.

Chapter 11 fitted a first line and chapter 12 a ridge; this chapter builds the model properly. The pieces are known; the craft is in the order, in choosing a target that suits a linear model, and in looking at the misses before anyone is told a number.

What you build: 4 tasks

Each task has tests. You write the code in the lab's editor and press Check.

  1. Task 1

    price_model.py: the price pipeline and its test MAE

    Write two functions in price_model.py.

    make_model() returns an unfitted TransformedTargetRegressor with func=np.log and inverse_func=np.exp, whose regressor is a Pipeline of a ColumnTransformer then Ridge(alpha=1.0). The ColumnTransformer sends condition to an OrdinalEncoder with categories=CONDITION (poor, fair, good, excellent, listed in the starter); TEXT (segment, branch, accident, service_records) to a OneHotEncoder(handle_unknown="ignore"); and NUMBERS (engine_cc, age_years, km_driven, owners) to a Pipeline of a SimpleImputer(strategy="constant", fill_value=0) then a RobustScaler.

    train_and_mae(X, y) is given X, bikes.csv's columns condition, TEXT and NUMBERS, and y, sold_price_rs. Split with train_test_split, test_size=0.25 and random_state=5, fit make_model() on the training rows and predict the test rows. Return two values separated by a comma: the fitted model, and its mean absolute error on the test rows in rupees (a float).

    The Check looks at every setting, fits a copy of your model on the training rows and compares its prices with the task's, then tries train_and_mae on all the bikes and on the first 1,000.

  2. Task 2

    within.py: the share of bikes priced within 10 %

    Write share_within(y_true, y_pred, tolerance) in within.py. y_true is the bikes' real prices (a Series), y_pred the model's prices (a NumPy array, one per bike, in the same order), and tolerance a share such as 0.1 for 10 %.

    Return the share of bikes whose miss (predicted minus real, without its sign) is at most tolerance times that bike's own real price, as one float from 0 to 1. Compare the two by position: a Series and an array have different row labels. The Check tries the Task 1 model's test prices at 0.1 and 0.05, and 5 bikes of its own.

  3. Task 3

    segments.py: which segment the model prices worst

    Write mae_by_segment(segments, y_true, y_pred) in segments.py. segments is the test bikes' segment column (a Series), y_true their real prices (a Series with the same row labels) and y_pred the model's prices (a NumPy array in the same order).

    Return a pandas Series named mae_rs, labelled by segment, each value the mean absolute error in rupees of that segment's bikes (the mean of their misses without their signs), sorted from the largest error to the smallest, so the worst segment comes first. The Check tries the Task 1 model's test prices and 6 bikes of its own.

  4. Task 4 · interview question

    Interview: log_target.py, why the log of the price

    Raju heard that a linear model prices bikes better on the log of the price. His raw_against_log(X_train, X_test, y_train, y_test) in log_target.py fits two LinearRegressions, one on the prices and one on their log (np.log), and returns their mean absolute errors on the test rows. His log model's MAE comes out at about the average price of a bike, so he says the log is useless.

    The four arguments are a split of bikes: X_train and X_test DataFrames of number columns, y_train and y_test prices in rupees. Fix it so both errors are in rupees: return two floats separated by a comma, the raw model's test MAE, then the log model's. The Check tries age_years and owners of all the bikes, and age_years of the commuters alone.

    Tanvi will ask: what was wrong with Raju's second number, and why does a straight line fit the log of a used bike's price better than the price itself?

What you will learn

  • The split and the floor (chapters 2 and 11): a random split, the mean price, the median by segment.
  • The pipeline (7, 8, 9, 10, 12): condition in its true order, one-hot text, a blank engine as 0, RobustScaler for the typos, ridge.
  • The log of the price with TransformedTargetRegressor: a line on the log is a constant share lost each year.
  • A small search (19): ridge's alpha on folds of the training rows.
  • Diagnosis (11): misses by segment, the largest misses, the typos.
  • A report people use: MAE in rupees, and the share of bikes within 10 %.

More in Real work

Build "Capstone: Raju's price model, end to end" yourself

Dobara refurbishes used bikes in Indore and lends to its buyers. You build the models that decide whom to lend to and at what price to sell. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.

The first 2 chapters of Machine Learning are free. This one comes with the all labs plan.