Postgres with Drizzle · Chapter 35 of 40

The queue masters (pg-boss)

By the end of this chapter you can hand slow work to a background queue that lives in Postgres: send a job, run it later or on a schedule, retry it when it fails, and count what is waiting. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.

  • Part: Production Patterns
  • 6 tasks
  • PostgreSQL
  • In the all labs plan

The first 2 chapters of Postgres with Drizzle are free. This one comes with the all labs plan.

Why you need this

Meera wants a welcome email 30 seconds after someone signs up, a fresh feed built for them, and expired sessions cleaned up every six hours.

Inside the sign-up request, the member waits for an email server. A setTimeout forgets its work on a restart, and a separate queue such as Redis is one more system to run and back up. pg-boss keeps its jobs as rows in Postgres: they survive restarts, and workers take them with SKIP LOCKED, so two workers never get the same job.

What you build: 6 tasks

Each task has tests. You write the code in the lab's editor and press Check.

  1. Task 1

    setupPgBoss: start pg-boss and create the queues

    Write setupPgBoss(). It creates a PgBoss connected to the test database (the connection string in process.env.TEST_DATABASE_URL, the database getPool() uses), starts it, and creates the two queues this chapter sends to: welcome-email and session-cleanup.

    Return the started PgBoss instance. The check keeps it and passes it to the other tasks.

  2. Task 2

    enqueueWelcomeEmail: a job for 30 seconds from now

    Write enqueueWelcomeEmail(boss, userId, email). boss is the started PgBoss, userId a number and email a string. It sends a job to the welcome-email queue whose data is exactly an object with userId and email, and which may not start until 30 seconds after it is sent.

    Return the job's id, the string send gives back.

  3. Task 3

    processWelcomeEmails: a worker for welcome emails

    Write processWelcomeEmails(boss, handler). handler is an async function that takes one job. It registers a worker on the welcome-email queue. pg-boss calls the worker's callback with an array of jobs: the callback calls handler once for each job in it, awaiting each call.

    Return the worker id, the string work gives back.

  4. Task 4

    scheduleSessionCleanup: a job on a cron schedule

    Write scheduleSessionCleanup(boss, cronExpression). cronExpression is a string, like a cron for every six hours. It schedules the session-cleanup queue with that cron expression, an empty object as the data, and the time zone UTC.

    Return nothing. Afterwards: pgboss.schedule has one row named session-cleanup whose cron is cronExpression.

  5. Task 5

    enqueueWithRetry: a job that retries

    Write enqueueWithRetry(boss, queueName, data, retryLimit). queueName is a string, data an object and retryLimit a number. It creates the queue queueName (it may not exist yet), then sends data to it with the retry limit retryLimit, a retry delay of 1 second, and an expiry of 300 seconds.

    Return the job's id. Afterwards the job's retryLimit is retryLimit and its data is data.

  6. Task 6

    getQueueHealth: jobs in each state

    Write getQueueHealth(boss, queueName). queueName is a string. In one SQL query on pgboss.job through getPool(), it counts the jobs of that queue (name is queueName) in four states: created, active, completed and failed.

    Return one object (the first row) with exactly created, active, completed and failed, all numbers, not strings; a state with no jobs is 0. The check sends three jobs first, so created is at least 3.

What you will learn

  • start, createQueue and send: start pg-boss, create a queue, put a job on it, now or later.
  • work: a worker that receives jobs in batches.
  • Retries: what happens to a job that throws.
  • schedule and the job table: jobs from a cron expression, and counting jobs by state with SQL.

More in Production Patterns

Build The queue masters (pg-boss) yourself

MingleSphereQL grows up: its backend talks to Postgres through Drizzle. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.

The first 2 chapters of Postgres with Drizzle are free. This one comes with the all labs plan.