Node.js traps · Chapter 7 of 16

pipe() does not clean up

Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.

  • Part: The event loop
  • 3 tasks
  • Node.js
  • In the all labs plan

The first 2 chapters of Node.js traps are free. This one comes with the all labs plan.

Why you need this

A leaked descriptor makes no log line and uses almost no memory. The process has a limit on open files; each cancelled download takes one more, and the day it reaches the limit every open fails, for every customer, until a restart. In the lab, with the limit set to 64:

46 files open
Error: EMFILE: too many open files, open '/tmp/poster.webp'

(The other 18 were already in use by Node itself.)

What you build: 3 tasks

Each task has tests. You write the code in the lab's editor and press Check.

  1. Task 1

    repro.js: leave five files open on purpose

    Write reproduce() in repro.js. Make the stand-in files with createTicketFiles() from ticket-files.js. Then, five times, one after another: make a download the customer leaves after 40,000 bytes with createDownload({ stopAfter: 40_000 }), start sendTicket(files, name, res) from ticket-download.js with any ticket name, and wait until res emits 'close' (once from node:events). After the five, let one more turn of the event loop pass with setImmediate, and resolve to files.openCount().

    The check runs your reproduce() twice: beside the original ticket-download.js it must resolve to 5 (every cancelled download left its file open), and beside a fixed ticket-download.js to 0. Wait for 'close' on the response, not for sendTicket: the original does not return a Promise. No real waiting: reproduce() must finish within 2 seconds.

  2. Task 2

    ticket-download.js: every download closes its file

    Fix sendTicket(files, name, res) in ticket-download.js so the file and the response are closed together, however the download ends. When the download finishes, the customer gets every byte (131,072 for a whole ticket), the response ends and the file is closed. When the customer goes away early (the response closes), the file is closed too. When the file fails while it is read, the response is destroyed, the file is closed, and one line is logged: download: <name> stopped (<code>), for example download: T0023-H9.pdf stopped (EIO).

    sendTicket must return a Promise that resolves once the download has finished or stopped, and never rejects. The check uses its own stand-ins from ticket-files.js: it cancels downloads after 1,000, 20,000 and 100,000 bytes, sends a whole ticket, and fails a read halfway, and counts the open files after each.

  3. Task 3

    ticket-download.test.js: a test that would have caught it

    Write a vitest test file, ticket-download.test.js, that would have caught the leak before the busy Saturday. The check runs your file three times: beside the original ticket-download.js, where at least one test must fail; beside the fixed ticket-download.js, where every test must pass; and beside a wrong fix that adds an 'error' listener to each stream and still uses pipe(), where at least one test must fail.

    So use the stand-ins from ticket-files.js: cancel a download halfway with createDownload({ stopAfter }), wait for the response's 'close' and one more turn, and expect files.openCount() to be 0; and check the other side too: a download nobody cancels still gets all 131,072 bytes. Each test must finish within 3 seconds.

What you will learn

  • repro.js: leave five files open on purpose
  • ticket-download.js: every download closes its file
  • ticket-download.test.js: a test that would have caught it

More in The event loop

Build "pipe() does not clean up" yourself

Parda's booking backend has run on Node for a year, and Nikhil keeps handing you incidents where Node did exactly what the team told it. Each is a runtime trap interviews love to ask about. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.

The first 2 chapters of Node.js traps are free. This one comes with the all labs plan.