NumPy · Chapter 14 of 18

Statistics

By the end of this chapter you can choose between the mean and the median and say why, measure spread with np.var and np.std (and know what ddof changes), read percentiles and quantiles, count values into bins with np.histogram, take a weighted average with np.average, measure how two things move together with np.cov and np.corrcoef, add up groups with np.bincount, and turn readings into z-scores. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.

  • Part: Analysis
  • 4 tasks
  • NumPy
  • In the all labs plan

The first 2 chapters of NumPy are free. This one comes with the all labs plan.

Why you need this

DhoopGrid's questions now need one honest number for many: what does a typical panel make at noon, which panels are well below the panels beside them, what price is the plant really paid, and does the output follow the sun or the heat?

The tools you have give the wrong answer to most of them. .mean() of a row with one dead panel is pulled down by 0 W, so every healthy panel in that row looks a little better than "typical" and a weak one can hide. The plain mean of the exchange's prices is ₹4,740 a MWh, but the plant sells most of its energy at noon, when power is cheapest, so Meenal is paid far less than that. And Imran cannot tell from two columns of 2,880 numbers how closely the output follows the irradiance without a number that measures it. This chapter gives you those numbers, and NumPy computes each of them in one call over the whole array.

What you build: 4 tasks

Each task has tests. You write the code in the lab's editor and press Check.

  1. Task 1

    weak_panels.py: weak panels against their own row

    Write weak_panels(noon) in weak_panels.py. noon is one block's grid of output in watts, shape (rows, cols), such as the day's block 48 of shape (40, 60). A panel is weak when its reading is under 85 % of its own row's median at that block: strictly less than 0.85 times the median of its row. Return a bool array of the same shape as noon, True for every weak panel; a dead panel (0 W) is weak.

    Each row has its own median, not the whole grid's. The Check tries blocks 48, 44 and 56 (the cloud) of the day, and a grid of 2 rows by 5 columns whose two rows have very different levels.

  2. Task 2

    paid_price.py: the price DhoopGrid is really paid

    Write paid_price(kwh, price) in paid_price.py. kwh is the energy the plant sold in each block and price the exchange's price for that block in rupees a MWh, both of shape (n,). Return one number: the average price weighted by the energy, so each block's price counts as much as the kWh sold in it (the sum of price times kWh, divided by the sum of kWh).

    Some prices are missing (nan): leave those blocks out, both their price and their kWh. The plain mean of the prices is not the answer. The Check tries the 30 days of blocks_30d.csv (kWh is plant_kw times 0.25 h; five prices are missing), the last day alone, and 4 blocks with one price missing.

  3. Task 3

    correlation.py: what moves the plant's output

    Write correlations(irradiance, ambient, plant_kw) in correlation.py. The three arrays have the same shape (n,): irradiance in W/m², air temperature in °C and the plant's output in kW, one value a block. Return an array of 2 floats, shape (2,): first the correlation coefficient of irradiance with plant_kw, then that of ambient with plant_kw (Pearson's r, each between -1 and 1).

    The Check tries the 2,880 blocks of blocks_30d.csv, only the blocks with sunlight (irradiance above 0), and 5 made-up blocks.

  4. Task 4 · interview question

    Interview: typical.py, mean or median

    Rukmini's typical_output(readings) in typical.py gets readings in watts, of any shape, and should return one number: the output of a typical panel. For row 22 at noon (block 48), where 15 panels are dusty and one has a hot spot, it says 442.7 W, while 44 of the row's 60 panels make more than that. Fix it so the answer is the output of the panel in the middle: half the panels make at least as much and half at most as much, so a few dead or dusty panels hardly move it.

    The Check tries row 22 at noon, the whole grid of block 40, and a row of 5 panels with one dead.

    Imran will ask: mean or median for "a typical panel", and why?

What you will learn

  • np.median: the middle value, which a few outliers hardly move.
  • np.var and np.std: the spread, and ddof for a sample.
  • np.percentile and np.quantile: the value below which a share of the data falls.
  • np.histogram: counts in bins, with no plot.
  • np.average with weights: an average in which some values count more.
  • np.cov and np.corrcoef: how two arrays move together.
  • np.bincount: counts and sums by an integer label.
  • z-scores: how many standard deviations from the mean, with broadcasting.

More in Analysis

Build "Statistics" yourself

DhoopGrid runs a solar plant in the Thar desert with 2,400 panels. You turn its raw readings into answers. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.

The first 2 chapters of NumPy are free. This one comes with the all labs plan.