NumPy · Chapter 10 of 18
Ufuncs and speed
By the end of this chapter you can use NumPy's ufuncs, the functions that work element by element in C (np.sqrt, np.abs, np.round, np.floor, np.exp, np.log, np.sin, np.maximum, np.minimum and the operators themselves), say what dtype they give and what happens outside their domain, write results into an array you already have with out= and the in-place operators, compute only where a condition holds with where=, use the ufunc methods reduce, accumulate and outer, and measure code before making it fast. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.
- Part: Shapes and maths
- 4 tasks
- NumPy
- In the all labs plan
The first 2 chapters of NumPy are free. This one comes with the all labs plan.
Why you need this
A panel in the Thar sun runs much hotter than the air, and above 25 °C it loses about 0.4 % of its output for every degree. Rukmini wants the plant's expected output corrected for heat: for every reading, the degrees above 25, but never below 0, because a cool morning does not add output. Meenal compares what the plant made with what it promised the grid, block by block, and at night the promise is 0. And the nightly script that works out the penalties calls a Python function once for every block of the year, through np.vectorize, and it is the slowest part of the night.
With what you know, "the larger of two numbers, for every element" is a mask and two assignments, "round to one decimal" has no operator at all, and dividing by a schedule that is sometimes 0 prints warnings. NumPy has a whole family of functions made for this, the ufuncs, and they are also the reason NumPy is fast. This chapter meets them, and then measures what really makes NumPy code fast or slow.
What you build: 4 tasks
Each task has tests. You write the code in the lab's editor and press Check.
- Task 1
derate.py: hot panels make less
Write derated(power_w, ambient_c, irradiance_wm2) in derate.py. A panel's cells are at ambient_c + 0.03 * irradiance_wm2 degrees; the degrees over is that temperature minus 25, but never less than 0 (a cool panel gains nothing); the derated output is power_w * (1 - 0.004 * degrees over), rounded to 0.1 W (one decimal place). Return it as a float array of the shape of power_w. derated(500.0, 35.0, 1000.0) is 420.0: the cells are at 65 °C, 40 over, and 500 times 0.84 is 420.
The arguments may be arrays or single numbers, and they broadcast. The Check calls it with the noon grid, shape (40, 60), with one ambient temperature (33.0) and one irradiance (980.2); with the day's 96 block totals and an ambient temperature and an irradiance for each block; and with 4 readings on a cool morning, three of them with cells below 25 °C.
- Task 2
ratio.py: output against the schedule, with no warning
Write performance(plant_kw, schedule_kw) in ratio.py. Both are float arrays of the same shape, in kW. Return plant_kw divided by schedule_kw for every block whose schedule is above 0, and 0.0 for every block whose schedule is 0 (the night), as a float array of the same shape. For plant [50.0, 0.0, 30.0] and schedule [100.0, 0.0, 0.0] it is [0.5 0. 0. ].
The Check calls it with 30 days of blocks from blocks_30d.csv as shape (2880,) and as (30, 96), and with 4 blocks where one has output but a schedule of 0. It treats any warning as an error: dividing by 0 gives a RuntimeWarning even when that answer is thrown away afterwards, so divide only where the schedule is above 0. No element of the result may be nan.
- Task 3
peak_so_far.py: the highest output so far
Write peak_so_far(kw) in peak_so_far.py. kw is the plant's output in kW, either one day's blocks, shape (96,), or many days, shape (days, 96). Return, at each block, the largest value from the start of that day up to and including that block, as a float array of the same shape. peak_so_far of [3.0, 1.0, 4.0, 1.0, 5.0] is [3. 3. 4. 4. 5.].
The Check calls it with the day's 96 blocks, with 30 days of 96 blocks (shape (30, 96): each day starts again from its own first block, so go along the last axis, not down the days), and with 6 blocks. Use no Python loop: a for, a while or a comprehension in peak_so_far fails the task.
- Task 4 · interview question
Interview: penalty.py, np.vectorize is not fast
The plant pays a penalty for every block where it made less than 90 % of its schedule: the kW short of 90 % of the schedule, times 0.25 h, times ₹3 a kWh; in other blocks it pays 0. The starter's block_penalty(plant_kw, schedule_kw) gives the right penalties, as a float array the shape of its inputs, through np.vectorize, and it is far too slow for a year of blocks. For plant [95.0, 80.0] and schedule [100.0, 100.0] it is [0. 7.5].
Rewrite block_penalty with whole-array maths so it gives the same numbers, with no np.vectorize anywhere in penalty.py and no Python loop in block_penalty. The Check compares the results for 30 days of blocks and for 5 blocks, then times it on 288,000 blocks: it must take under 25 ms (whole-array maths takes about 1 ms).
Imran will ask: np.vectorize takes the whole array, so why is it no faster than a loop?
What you will learn
- Ufuncs: functions that work element by element in one C loop, with one input (np.sqrt, np.abs, np.round, np.floor, np.exp, np.log, np.sin) or two (np.maximum, np.minimum, np.hypot, and np.add, which is +).
- The result's dtype, and nan with a RuntimeWarning outside a function's domain; np.errstate to silence it on purpose.
- out= and the in-place operators (a *= 2), and the casting error when a float goes into an int array.
- where=: compute only where a condition is True.
- reduce, accumulate and outer: what every two-input ufunc can do.
- np.vectorize is a Python loop, measured.
- Making it fast: measure first, no Python loops, no needless new arrays, a contiguous layout.
More in Shapes and maths
Build "Ufuncs and speed" yourself
DhoopGrid runs a solar plant in the Thar desert with 2,400 panels. You turn its raw readings into answers. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.
The first 2 chapters of NumPy are free. This one comes with the all labs plan.