Machine Learning · Chapter 22 of 26

PCA, twelve measurements in a few numbers

By the end of this chapter you can find the few directions that a table of correlated measurements really varies along, say how much of the variation each one holds, turn every bike into one or two scores, and measure what a model loses when it learns from those scores instead of the columns. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.

  • Part: Tuning
  • 4 tasks
  • Python
  • In the all labs plan

The first 2 chapters of Machine Learning are free. This one comes with the all labs plan.

Why you need this

When a used bike comes into Dobara's workshop, Raju measures twelve things before it is priced: the tread on both tyres, both brake pads, the battery's voltage, the headlight's brightness, the horn, the suspension's travel, the paint and the seat (graded 1 to 10), the rust spots, and how loud it is at idle. The sheet goes into inspection.csv, one row a bike, 1,600 bikes.

Rachna wants one number on the sheet that says how worn a bike is, so a buyer and a branch manager can compare two bikes at a glance. Twelve numbers are too many to read, and they are not twelve separate facts: a bike with thin tyres usually has thin brake pads, a dim headlight and more rust too. They move together.

What you know so far cannot make that number well. Picking one column, say the front tyre, throws away the other eleven. Adding the twelve up is meaningless: the headlight is measured in hundreds of lux and the tyres in a few millimetres, so the sum is almost all headlight, and rust counts the wrong way round. What you want is the best weighted sum: the weights that capture as much of how bikes differ as one number can. Finding those weights is principal component analysis.

What you build: 4 tasks

Each task has tests. You write the code in the lab's editor and press Check.

  1. Task 1

    variance.py: how much each component explains

    Write two functions in variance.py.

    variance_shares(measurements) is given a DataFrame of numbers only, one row a bike, like inspection.csv without its bike_id column. Make a Pipeline of a StandardScaler then a PCA with no n_components (so it keeps every component), and fit it on every row: PCA learns from the table alone, with no label. Return two values separated by a comma: the fitted pipeline, and the share of the variance each component explains, the PCA step's explained_variance_ratio_ (a NumPy array, in the components' order).

    components_needed(shares, target) is given such an array (largest first, adding up to 1) and a target share from 0 to 1. Return the smallest number of first components whose shares add up to target or more, as a whole number: with shares [0.5, 0.3, 0.2], target 0.75 gives 2 and target 0.85 gives 3.

    The Check tries the twelve measurements of the first 1,000 bikes, a made-up table of 5 columns, and several shares and targets.

  2. Task 2

    wear.py: a wear score for every bike

    Write wear_scores(measurements) in wear.py. measurements is a DataFrame of inspection numbers, one row a bike, with a column rust_spots among them (like inspection.csv without bike_id). Scale the columns with a StandardScaler and fit a PCA on every row (a Pipeline of the two is easiest), and take each bike's score on the first component.

    A component's sign is arbitrary, so choose it: if the first component's weight for rust_spots is below 0, multiply every score by -1, so that a more worn bike scores higher.

    Return a pandas Series named wear, one score per bike, with the row labels of measurements. The Check tries the first 1,000 bikes, the bikes from B1001 on, and a made-up table of 3 columns where PCA's own sign comes out the other way.

  3. Task 3

    price_k.py: a price model on k components

    Write r2_by_k(bikes, ks) in price_k.py. bikes is bikes.csv merged with inspection.csv on bike_id (one row a bike, the columns of both files); ks is a list of whole numbers. X is the columns segment and MEASUREMENTS (the twelve measurements, listed in the starter), y is sold_price_rs. Split with train_test_split, test_size=0.25 and random_state=3.

    For each k in ks, make a Pipeline of a ColumnTransformer then a LinearRegression. The ColumnTransformer sends segment to a OneHotEncoder and MEASUREMENTS to a Pipeline of a StandardScaler then PCA(n_components=k). Fit it on the training rows, and take its R² on the test rows with the model's own score method.

    Return a dict with each k of ks as a key and its test R² (a float) as the value. The Check tries ks [1, 2, 12] and [3, 5] on all the bikes, and [2, 4] on the first 800 bikes.

  4. Task 4 · interview question

    Interview: unscaled.py, PCA found only headlights

    Raju ran PCA on the inspection to see which measurements move together. His first_weights(measurements) returns the first component's weight for each column, and it says the first component is headlight_lux and almost nothing else. measurements is a DataFrame of numbers only, one row a bike, like inspection.csv without bike_id.

    Fix it so that the columns are scaled with a StandardScaler before the PCA is fitted on every row, and return the first component's weights as a pandas Series labelled by the columns of measurements, in their order. A component's sign is arbitrary, so the Check accepts the weights or all of them times -1. It tries the twelve measurements of the first 1,000 bikes and a made-up table of 4 columns in very different units.

    Tanvi will ask: why did PCA without scaling find only the headlights, and what does scaling change about the question PCA answers?

What you will learn

  • PCA: finds the directions a table varies along most, without a label.
  • components_: each direction as a weight for every column; a bike's score is the weighted sum.
  • explained_variance_ratio_: the share of the variation each component holds.
  • Scaling first: without it, the column in the biggest units wins.
  • n_components: keep a number of components, or enough for a share.
  • transform and inverse_transform: from columns to scores and back, and what is lost.

More in Tuning

Build "PCA, twelve measurements in a few numbers" yourself

Dobara refurbishes used bikes in Indore and lends to its buyers. You build the models that decide whom to lend to and at what price to sell. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.

The first 2 chapters of Machine Learning are free. This one comes with the all labs plan.