Machine Learning · Chapter 12 of 26
Ridge and lasso
By the end of this chapter you can say why many correlated columns make a linear model's coefficients big and unstable, add a penalty that shrinks them (Ridge) or sets some to exactly zero (Lasso), explain why the penalty needs scaled columns, and let RidgeCV and LassoCV choose the penalty's strength from the training rows. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.
- Part: Regression
- 4 tasks
- Python
- In the all labs plan
The first 2 chapters of Machine Learning are free. This one comes with the all labs plan.
Why you need this
Every bike that comes into the workshop gets Raju's inspection: twelve measurements, from tyre tread and brake pads to the battery's voltage, the paint and the rust spots. It takes forty minutes a bike. Rachna's question is blunt: which of those twelve numbers change what a bike sells for, and which could the workshop stop measuring?
The obvious answer is chapter 11's: put them all in a linear regression and read the coefficients. But the measurements move together (worn tyres come with worn brakes), and when columns move together a linear regression cannot tell which one deserves the credit. It splits it almost at random: refit on another sample of bikes and a coefficient changes size or even sign. A sentence like "each rust spot takes ₹381 off" is worthless if the next sample says "adds ₹260". You need a model that keeps its coefficients small and steady, and one that can say "this column does not matter" with a clean zero.
What you build: 4 tasks
Each task has tests. You write the code in the lab's editor and press Check.
- Task 1
shrink.py: how much ridge shrinks
Every task in this chapter is given X, a DataFrame of numeric columns of the bikes (bikes.csv merged with inspection.csv by bike_id: age_years, km_driven, owners, the twelve measurements, and segment, condition, accident and service_records one-hot encoded), and y, their sold_price_rs.
Write coef_sizes(X, y, alphas) in shrink.py. alphas is a list of numbers. For each alpha, in the list's order, make a Pipeline of a StandardScaler then a Ridge with that alpha (make_pipeline is fine), fit it on all the rows of X and y (no split: this task looks at the coefficients, not a score), and add up the absolute values of the Ridge's coef_.
Return a DataFrame with one row per alpha, in the list's order, and exactly two columns: "alpha", the alpha, and "total_size", that sum as a float. The coefficients are the Ridge's, the pipeline's last step.
The Check tries the bikes with two lists of alphas, and 120 made-up bikes.
- Task 2
lasso_keeps.py: the columns lasso keeps
Write kept_columns(X, y, alpha) in lasso_keeps.py. X and y are the bikes as in Task 1, alpha one number. Make a Pipeline of a StandardScaler then a Lasso with this alpha (its other settings at their defaults), fit it on all the rows of X and y, and find the columns whose Lasso coefficient is not 0.
Return a list of those column names (strings), in X's column order. Lasso sets a coefficient to exactly 0, so compare with 0 itself, not with a small number.
The Check tries the bikes at alphas 300 and 3000, the bikes with their columns in another order at alpha 1000, and 120 made-up bikes.
- Task 3
tuned_ridge.py: let RidgeCV choose alpha
Write tuned_ridge(X, y) in tuned_ridge.py. X and y are the bikes as in Task 1. Split the rows with train_test_split, test_size=0.25 and random_state=8. Make a Pipeline of a StandardScaler then a RidgeCV with alphas=[0.01, 0.1, 1, 10, 100, 1000] (its other settings at their defaults), and fit it on the training rows only.
Return three values separated by commas: the fitted Pipeline, the alpha RidgeCV chose (a float), and the pipeline's R² on the test rows (a float).
RidgeCV chooses alpha from the training rows alone; the test rows are used once, for the R² at the end. The Check fits a fresh copy of your pipeline on the same training rows and compares the two. It tries the bikes and 200 made-up bikes.
- Task 4 · interview question
Interview: fair_lasso.py, a penalty per what?
Raju fitted a Lasso with alpha=1000 straight on X and y (the bikes as in Task 1) and told Rachna which columns matter. It kept km_driven and headlight_lux, measured in big units, and dropped accident_yes and condition_poor, which change a price by thousands of rupees. Fix fair_lasso(X, y) in fair_lasso.py so that it still fits Raju's Lasso(alpha=1000) on X and y as they are, on all the rows, and also fits a Pipeline of a StandardScaler then a Lasso(alpha=1000) on all the rows.
Return two values separated by a comma: the fitted Pipeline, and a list of the columns the scaled Lasso keeps (a coefficient that is not 0) but Raju's dropped (a coefficient of exactly 0), in X's column order. The Check tries the bikes and 150 made-up bikes.
Tanvi will ask: why does the penalty need scaled columns?
What you will learn
- Correlated features: why they make coefficients big and unstable.
- Ridge: least squares plus a price on the squared coefficients; alpha sets the price, and every coefficient shrinks toward 0.
- Lasso: a price on the absolute coefficients; some become exactly 0, which picks columns for you.
- Scale first: a penalty in rupees per unit is unfair to columns in small units.
- RidgeCV and LassoCV: try a list of alphas on the training rows and keep the best, in alpha_.
- ElasticNet, and which one to reach for.
More in Regression
Build "Ridge and lasso" yourself
Dobara refurbishes used bikes in Indore and lends to its buyers. You build the models that decide whom to lend to and at what price to sell. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.
The first 2 chapters of Machine Learning are free. This one comes with the all labs plan.