Machine Learning · Chapter 24 of 26
Capstone: Farida's credit model, end to end
By the end of this chapter you have built Dobara's credit model the way a real one is built: framed in rupees, split by date before anything else, measured against Farida's rule, tuned and given its threshold on the building rows only, reported once on loans it never saw, and used on this week's applications, new dealers and all. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.
- Part: Real work
- 5 tasks
- Python
- In the all labs plan
The first 2 chapters of Machine Learning are free. This one comes with the all labs plan.
Why you need this
Every chapter so far fixed one piece. Now Rachna wants the whole thing, because Dobara will lend by it: if the model is wrong, it is wrong with real money, ₹30,000 for each default approved and ₹9,000 for each good buyer refused. Farida has decided loans by her rule for years, and she will switch only if the model beats it on loans neither of them has looked at. This week's 250 applications are waiting, and two of them came through dealers that did not exist when the past loans were made.
Nothing here is new. What is new is the order, and the discipline: each decision (which columns, which C, which threshold) is made on the building rows, and the final rows are opened once, at the end, for the report. Get the order wrong and every number in the report is a little too good, and nobody can tell by looking.
What you build: 5 tasks
Each task has tests. You write the code in the lab's editor and press Check.
- Task 1
date_split.py: split by date, before anything else
Write split_by_date(loans, cutoff) in date_split.py. loans is a DataFrame with the columns of loans.csv, as pd.read_csv reads it (do not change it); cutoff is a date as text, like "2025-04-01". The loans whose applied_on is before the cutoff are the building rows; those on the cutoff or after are the final test rows.
Return four values separated by a comma, X_build, X_final, y_build, y_final: the two X parts are DataFrames of exactly the FEATURES columns listed in the starter, in that order (every column but loan_id, applied_on, collection_calls and defaulted); the two y parts are the column defaulted as Series. Every part keeps the table's row order and row labels.
applied_on is text like "2025-04-01", and text dates in this form compare in date order. The Check tries the cutoffs "2025-04-01" and "2025-01-01" on all the loans, and "2025-05-15" on the last 600.
- Task 2
credit_model.py: the full pipeline, tuned on the building rows
Write two functions in credit_model.py.
make_model() returns an unfitted Pipeline, made with make_pipeline, of a ColumnTransformer then a LogisticRegression(max_iter=1000). The ColumnTransformer sends NUMBERS (listed in the starter) to a Pipeline of a SimpleImputer(strategy="median", add_indicator=True) then a StandardScaler, and TEXT (listed in the starter) to a OneHotEncoder(handle_unknown="ignore", min_frequency=10).
tune(X, y) is given the building rows, X_build and y_build of Task 1 with the cutoff "2025-04-01". Make a GridSearchCV of make_model() with the grid {"logisticregression__C": [0.01, 0.1, 1.0]}, cv=StratifiedKFold(5, shuffle=True, random_state=0) and scoring="roc_auc", fit it on X and y, and return the fitted GridSearchCV.
The Check looks at every setting, fits a copy of your pipeline on the building rows and compares its probabilities with the task's, then compares your search's best_params_, best_score_ and best_estimator_ with its own.
- Task 3
threshold.py: the cheapest threshold, from Farida's rupees
Write cheapest_threshold(model, X, y, thresholds) in threshold.py. model is an unfitted pipeline (Task 2's make_model() with the C the search chose), X and y are the building rows, and thresholds is a list of cuts from 0 to 1.
Get every building row's out-of-fold probability of default with cross_val_predict, cv=StratifiedKFold(5, shuffle=True, random_state=0) and method="predict_proba", taking the column for label 1. For each threshold t, a loan is refused when its probability is t or more; its cost is 30,000 rupees for each default approved (a false negative) plus 9,000 for each good buyer refused (a false positive).
Return two values separated by a comma: the threshold with the smallest cost (on a tie, the first one in the list), and that cost in rupees as a whole number. The Check tries two lists of thresholds with C=0.1.
- Task 4 · interview question
Interview: final_report.py, a report that looks too good
Tanvi's draft of final_report(model, threshold, X_final, y_final) in final_report.py reports an AUC on the final rows higher than any cross-validated score the search saw. Rachna is pleased; Tanvi is not.
model is the search's best pipeline, already fitted on the building rows; threshold is Task 3's; X_final and y_final are Task 1's final rows. Fix it so the final rows are used once, to measure the model as it is given: its probability of default for each final row, and a loan refused when that probability is threshold or more.
Return a dict with exactly the keys "auc" (roc_auc_score of the probabilities), "recall" (of the defaults, label 1, the share refused), "precision" (of the loans refused, the share that defaulted) and "cost_rs" (30,000 a default approved plus 9,000 a good buyer refused, a whole number). The Check gives its own fitted model and the thresholds 0.25 and 0.4.
Tanvi will ask: why was the draft's AUC so good, and why are the final rows used only once, at the very end?
- Task 5
decide.py: this week's applications
Write decide(model, threshold, applications) in decide.py. model is the final pipeline, fitted on the building rows; threshold is Task 3's; applications is applications_new.csv as pd.read_csv reads it (250 rows, no defaulted column, and two dealers, D37 and D38, that no past loan has).
Return a DataFrame with the row labels of applications and exactly these columns, in this order: loan_id (the application's loan_id), p_default (the model's probability of default, label 1) and decision (the text "refuse" when p_default is threshold or more, else "approve"). Nothing is refitted on the applications. The Check tries this week's 250 at 0.25, and 3 applications of its own (one from D38, one with a blank bureau score) at 0.1.
What you will learn
- Framing (chapters 1 and 3): who decides, the label, the day of the prediction, the rupees of each mistake.
- The split (2): by date, because next quarter is what the model will meet.
- Baselines (1 and 3): Farida's rule and "approve everyone", in rupees.
- The pipeline (8, 9, 10): imputer with indicator, scaler, one-hot encoding that survives new dealers.
- The search and the threshold (4, 5, 19): cross-validation on the building rows only.
- The report and the decisions (3, 4): the final rows once, then the applications.
More in Real work
Build "Capstone: Farida's credit model, end to end" yourself
Dobara refurbishes used bikes in Indore and lends to its buyers. You build the models that decide whom to lend to and at what price to sell. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.
The first 2 chapters of Machine Learning are free. This one comes with the all labs plan.