Python traps 2 · Chapter 13 of 16

Errors that vanish in a thread pool

Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.

  • Part: Libraries
  • 3 tasks
  • Python
  • In the all labs plan

The first 2 chapters of Python traps 2 are free. This one comes with the all labs plan.

What you build: 3 tasks

Each task has tests. You write the code in the lab's editor and press Check.

  1. Task 1

    repro.py: the invoice that failed and was counted as sent

    Write reproduce() in repro.py. Make three invoices, each a dict with "id", "account" and "amount", with the ids "INV-301", "INV-302" and "INV-303", in that order, and a fake send(invoice) that raises ConnectionError for INV-302 and does nothing for the others. Call send_all from invoices.py with the three invoices and your fake, and return what it gave.

    The check runs your reproduce() twice, each time in a fresh python3: beside the original invoices.py it must return {"sent": ["INV-301", "INV-302", "INV-303"], "failed": []} (INV-302 counted as sent), and beside a fixed invoices.py it must return {"sent": ["INV-301", "INV-303"], "failed": ["INV-302"]}. So your fake must really raise for INV-302 alone. No real waiting and no input(): reproduce() must finish within 3 seconds.

  2. Task 2

    invoices.py: every invoice is sent or reported

    Fix send_all(invoices, send, workers=4) in invoices.py. It returns {"sent": [ids], "failed": [ids]}, with every invoice's id in exactly one of the two lists, both in the order the invoices came in: an invoice whose send returns is sent, one whose send raises (any exception) is failed, and a failure never stops the other invoices from being sent or reported.

    Keep sending on a ThreadPoolExecutor with workers threads, every invoice tried once, and keep summary as it is. The check imports a fresh copy of your invoices.py and uses its own fake send, which raises for chosen invoices (the first, the middle, the last, all of them, with different exceptions) and records every invoice it was given; in one run its first send finishes last, so the lists must follow the input, not the order the threads finish in.

  3. Task 3

    test_invoices.py: a test that would have caught it

    Write a pytest file, test_invoices.py, that would have caught this bug before the monthly run. The check runs your file three times: beside the original invoices.py, where at least one test must fail; beside the fixed invoices.py, where every test must pass; and beside a wrong fix that reads pool.map's results inside one try, so the first failure stops the loop, where at least one test must fail.

    So use a fake send that raises for one invoice in the middle of a batch, and check the whole result: the failed one is in "failed", and the invoices before and after it are in "sent". No real waiting: each test must finish within 3 seconds.

What you will learn

  • repro.py: the invoice that failed and was counted as sent
  • invoices.py: every invoice is sent or reported
  • test_invoices.py: a test that would have caught it

More in Libraries

Build "Errors that vanish in a thread pool" yourself

Monsoon season at DabbaDrop brings a new pile of tickets from Kunal. Each is a real trap interviews love to ask about. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.

The first 2 chapters of Python traps 2 are free. This one comes with the all labs plan.