NumPy · Chapter 11 of 18

Joining and splitting

By the end of this chapter you can join arrays along an axis they already have with np.concatenate or on a new axis with np.stack, use the shortcuts np.vstack, np.hstack, np.column_stack and np.dstack, say why np.append in a loop gets slower and slower, cut an array into equal or uneven pieces with np.split, np.array_split, np.hsplit and np.vsplit, and repeat an array's elements with np.repeat or the whole array with np.tile. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.

  • Part: Shapes and maths
  • 4 tasks
  • NumPy
  • In the all labs plan

The first 2 chapters of NumPy are free. This one comes with the all labs plan.

Why you need this

DhoopGrid's data rarely arrives as one array. Rukmini's weekly report starts from seven separate days. Bhanwar works one block of the plant at a time, and each inverter looks after its own grid of 10 rows by 20 columns, so the plant's (40, 60) grid has to be cut into the pieces he works on. Meenal's planning sheet has one schedule value an hour, and the grid wants one every 15 minutes. And the nightly script builds its long arrays by appending one day at a time, which gets slower every night the file grows.

With what you know, you can slice one piece out at a time, grid[0:10], grid[10:20], and type the numbers in for every piece, or loop over the days and write each into a big array of zeros at the right offset. Both work and both break the day the plant gets a fifth block. NumPy has functions that join a list of arrays in one step and cut one array into a list of pieces, and they work out the sizes themselves.

What you build: 4 tasks

Each task has tests. You write the code in the lab's editor and press Check.

  1. Task 1

    week.py: a week from seven days

    Write two functions in week.py. Each takes days, a Python list of 1-D float arrays of the same length, each one day's plant output in kW, one value a block.

    week_series(days) returns the days end to end as one 1-D float array: for 7 days of 96 blocks, shape (672,), day 0's blocks then day 1's and so on.

    week_table(days) returns a 2-D float array with one row a day: for 7 days of 96 blocks, shape (7, 96).

    The Check calls both with the last 7 days of blocks_30d.csv (96 blocks each) and with 3 days of 4 blocks.

  2. Task 2

    split_plant.py: the plant in pieces

    Write two functions in split_plant.py. Each takes grid, the plant as a 2-D array of 40 rows of 60 panels, shape (40, 60), and returns a Python list of arrays.

    plant_blocks(grid) returns the 4 blocks A to D, in order: rows 0 to 9, 10 to 19, 20 to 29 and 30 to 39, each of shape (10, 60).

    inverter_grids(grid) returns the 12 inverter grids INV01 to INV12, in order, each of shape (10, 20): every block's rows cut into three groups of 20 columns, block A's three first (INV01 is rows 0 to 9 with columns 0 to 19, INV02 the same rows with columns 20 to 39, INV03 columns 40 to 59), then block B's, and so on.

    The Check calls both with the noon grid, with the panel numbers 1 to 2400 as a (40, 60) grid, and with a small plant of shape (8, 6), whose blocks are 2 rows of 6 and whose inverter grids are 2 by 2. So always cut into 4 equal groups of rows and 3 equal groups of columns, and never type in 10 or 20.

  3. Task 3

    hourly.py: hours to blocks

    Write two functions in hourly.py.

    to_blocks(hourly) takes a 1-D float array of one value an hour and returns each value four times in a row (a block is 15 minutes), as a 1-D float array four times as long: to_blocks of [0.0, 350.0] is [ 0. 0. 0. 0. 350. 350. 350. 350.].

    repeat_days(profile, n_days) takes one day's values, a 1-D float array, and returns the whole profile n_days times end to end, as a 1-D float array: for [1.0, 2.0] and 2 days, [1. 2. 1. 2.].

    The Check calls to_blocks with 24 hours and with 3 hours, and repeat_days with a profile of 96 blocks for 7 days and a profile of 3 blocks for 2 days and for 1 day.

  4. Task 4 · interview question

    Interview: collect.py, appending in a loop

    The nightly script's collect(days) takes a Python list of 1-D float arrays (of any lengths) and returns them end to end as one 1-D float array: collect of [1.5, 2.0] and [0.0] is [1.5 2. 0. ]. It gives the right answer by appending each day in a loop with np.append, and it gets slower and slower as the list grows.

    Rewrite it to join the days in one step, with no np.append anywhere in collect.py. The Check compares the results for 30 days of 96 blocks and for 3 pieces of 2, 1 and 3 blocks, then times it on 800 pieces of 2,400 readings: it must take under 50 ms (one join takes about 1 ms).

    Imran will ask: why does appending in a loop get slower the longer the array is?

What you will learn

  • np.concatenate: join arrays end to end along an axis they already have.
  • np.stack: join arrays of the same shape on a new axis.
  • np.vstack, np.hstack, np.column_stack, np.dstack: shortcuts for the common cases.
  • np.append makes a full copy every time: a loop of appends, measured.
  • np.split, np.array_split, np.hsplit, np.vsplit: one array into a list of pieces, equal or uneven.
  • np.repeat against np.tile: each element again, or the whole array again.

More in Shapes and maths

Build "Joining and splitting" yourself

DhoopGrid runs a solar plant in the Thar desert with 2,400 panels. You turn its raw readings into answers. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.

The first 2 chapters of NumPy are free. This one comes with the all labs plan.