NumPy · Chapter 6 of 18

Fancy indexing

By the end of this chapter you can pick elements of an array by a list of positions, in any order and with repeats, turn panel numbers into rows and columns and back, take every listed row with every listed column with np.ix_, look values up in a table with np.take, and add at repeated positions correctly with np.add.at. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.

  • Part: Arrays from zero
  • 4 tasks
  • NumPy
  • In the all labs plan

The first 2 chapters of NumPy are free. This one comes with the all labs plan.

Why you need this

Every morning Bhanwar gets a list from the overnight script: the panels to look at today, by number, in the order he will walk the rows. P1217, P1273, P1336. He wants each one's noon output beside its number, so he knows what "weak" means before he gets there. Meenal gets the week's fault reports, one line a fault, and wants the count for each of the twelve inverters, because a fault on an inverter's string costs more than a fault on one panel.

What you know so far does not fit. A slice picks a run of neighbours, and Bhanwar's panels are scattered over the plant. A mask from chapter 5 picks by a condition, always in the array's own order, and each element at most once; Bhanwar's list has its own order, and a panel can be on it twice. With the panels' rows and columns in two arrays, you could loop and pick one panel at a time:

watts = []
for r, c in zip(rows, cols):
    watts.append(grid[r, c])

That works, and it is Python work for every panel again: about 24 times slower than one NumPy indexing on 800 panels, measured below. NumPy can take the whole list of positions at once.

What you build: 4 tasks

Each task has tests. You write the code in the lab's editor and press Check.

  1. Task 1

    noon_by_id.py: Bhanwar's panels at noon

    Bhanwar's list is panel numbers: P1273 is the number 1273. Write noon_watts(day, numbers) in noon_by_id.py. day is the day's array of shape (96, 40, 60); numbers is an int array of panel numbers from 1 to 2400, in the order he will walk them, and a number may appear more than once. Panel number n sits at row (n - 1) // 60 and column (n - 1) % 60.

    Return every listed panel's output at noon (block 48), in watts, as a float array with one value per number, in the same order: for numbers [1, 61, 1] it has shape (3,), row 0 column 0, then row 1 column 0, then row 0 column 0 again.

    The Check tries [1273, 1, 2400, 60, 61, 1273], [462], 50 other numbers, and a made-up day where every reading is different. Use no Python loop: a for, a while or a comprehension in noon_watts fails the task.

  2. Task 2

    clean_grid.py: the patch Bhanwar will clean

    Write cleaning_grid(grid, rows, cols) in clean_grid.py. grid is one block's readings of shape (40, 60); rows and cols are int arrays of row and column numbers. Return every listed row with every listed column: an array of shape (len(rows), len(cols)) where [i, j] is grid[rows[i], cols[j]].

    The Check tries 3 rows by 4 columns (rows 22 to 24, columns 30 to 33, the dusty patch), 1 row by 2 columns ([5] and [0, 59]), and 4 rows by 4 columns ([39, 0, 17, 2] and [7, 8, 9, 10]), where pairing the two arrays up gives only 4 panels and no error.

  3. Task 3

    inverter_lookup.py: each fault's inverter

    Imran keeps a map of shape (40, 60): inverter_map[r, c] is the inverter number, 1 to 12, of the panel at row r and column c. Write inverter_of(inverter_map, numbers) in inverter_lookup.py. numbers is an int array of panel numbers from 1 to 2400; panel n is at the flat position n - 1 of the grid, counting along each row.

    Return the inverter number of every reported panel, read from the map, as an int array of the same shape as numbers and in the same order.

    The Check tries [1, 20, 21, 2400, 1273, 1273, 462] and 40 other numbers with DhoopGrid's map, and also passes another map with other numbers in it, so a formula for DhoopGrid's own layout fails: read the map. Use no Python loop.

  4. Task 4 · interview question

    Interview: fault_count.py, the faults that went missing

    faults_per_inverter(inverters) in fault_count.py takes an int array with one inverter number, 1 to 12, for each fault report of the week, and returns how many reports each inverter had, as an int array of shape (12,): [0] is INV01's count and [11] is INV12's. For [8, 8, 3] it is 2 at [7], 1 at [2] and 0 everywhere else.

    Bhanwar logged three faults on INV08, and the function says 1. Fix it so it counts every report. The Check tries [8, 8, 8, 1, 12, 3, 8], 200 reports, [5], and an empty array (twelve zeros). Use no Python loop: a for, a while or a comprehension fails the task.

    Imran will ask: why did counts[ids] += 1 count each inverter only once, and what adds once for every repeat?

What you will learn

  • An index array: a list or an int array inside the brackets picks those positions, in that order, repeats included.
  • Panel numbers to rows and columns: // and %, np.unravel_index, and np.ravel_multi_index back.
  • Two index arrays pair up: the first row with the first column, not every row with every column.
  • np.ix_: every listed row with every listed column, a grid.
  • Always a copy: fancy indexing never gives a view.
  • np.take and lookup tables: index a small table with a big array of codes.
  • The repeated-index trap: a[idx] += 1 adds once per position, and np.add.at adds once per repeat.

More in Arrays from zero

Build "Fancy indexing" yourself

DhoopGrid runs a solar plant in the Thar desert with 2,400 panels. You turn its raw readings into answers. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.

The first 2 chapters of NumPy are free. This one comes with the all labs plan.