Redis · Chapter 20 of 26

Designing the keyspace

By the end of this chapter you can name every GullyXI key from one module with a version in it, choose between one hash and many keys by measuring them, build an index with sets, find the biggest keys without blocking Redis, and keep a hot key from melting the one server that holds it. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.

  • Part: Redis in production
  • 4 tasks
  • Redis
  • In the all labs plan

The first 2 chapters of Redis are free. This one comes with the all labs plan.

Why you need this

GullyXI's Redis now holds everything from the last nineteen chapters: the match centre, logins, sessions, leaderboards, queues and streams. Each chapter chose its own key names, and it shows. Gurpreet found fan:u1002 holding a string from an old feature while the new profile code wanted a hash there; her memory graph jumped by 40 MB after a deploy and nobody could say which keys did it; and at the toss one key, the opener's match centre, took more reads than the rest of Redis together.

Redis has no tables and no schema. A key is just a name, and every key lives in one big space of names, so the names are the design: they decide what collides, what you can find, what you can delete, and what you can move. Farhan wants the design written down, in code, before the next match night.

What you build: 4 tasks

Each task has tests. You write the code in the lab's editor and press Check.

  1. Task 1

    keys.js: a module of key names

    Write makeKeys(version) in keys.js, with version v1 when it is left out. It returns an object of four functions that make key names: gx, the version, then the parts, joined by colons. fan('u1001') makes gx:v1:fan:u1001, city('Pune') makes gx:v1:fans:city:pune (the city in lower case), page(2001) makes gx:v1:page:match:2001 and board(71) makes gx:v1:board:contest:71; with makeKeys('v2'), page(2003) makes gx:v2:page:match:2003.

    An id or a city that is empty or missing, or holds a colon, a space or a star, throws an Error. So does a version that is not the letter v followed by a number (1 and v1:x both throw).

  2. Task 2

    city-index.js: fans by city

    In city-index.js, saveFans(redis, fans) saves each fan ({ id, name, handle, city }) as the hash gx:v1:fan:<id> with the fields name, handle and city, and adds the id to the set gx:v1:fans:city:<city in lower case>. It makes no other keys and returns how many fans it saved (400 for GullyXI's fans, which the Check passes).

    fansIn(redis, city) returns the ids in that city's set, sorted, for a city in any case, and [] for a city with no fans. It reads the set only: no SCAN, no KEYS, no HGETALL.

    moveFan(redis, fanId, newCity) changes the fan's city field and moves the id from the old city's set to the new one's, and returns true. A move to the fan's own city leaves the fan in it. A fan with no hash returns false and changes nothing.

  3. Task 3

    biggest.js: find the biggest keys

    In biggest.js, biggestKeys(redis, pattern, n) walks every key that matches pattern with SCAN (never KEYS), asks MEMORY USAGE for each with its default sampling, and returns the n biggest as a list of { key, bytes }, biggest first, equal sizes in key order. Each key appears once even when SCAN hands it out twice; a pattern with no keys returns [].

    The Check fills a Redis with fan hashes, a 3,000-item list, a big set, a sorted set, a page and keys under another prefix, and compares your list with MEMORY USAGE of every gx:v1 key.

  4. Task 4 · interview question

    Interview: hot-key.js, a hot key on match night

    In hot-key.js, makePageReader(redis, ttlMs) (ttlMs 1000 when left out) returns readPage(matchId, now), with now in milliseconds and Date.now() when left out. readPage answers the text of gx:v1:page:match:<matchId>, or null when the key does not exist.

    Each reader keeps its own copy of each page for ttlMs: 200 reads at once at the same now send one GET; a read at now + 999 sends none; a read at now + 1000 asks Redis again. A missing page is never kept, so every read of it asks Redis. Two readers (two app servers) each ask Redis for their own copy.

    Farhan will ask: why does a hot key not get faster when you add Redis servers, and what does each local copy cost you?

What you will learn

  • A naming scheme with a version: gx:v1: in front of every key, made by one module.
  • OBJECT ENCODING and MEMORY USAGE: how Redis stores a key, and its bytes.
  • An index with sets: a second copy of one fact, so one read answers a question.
  • Big keys: finding them with SCAN and MEMORY USAGE, and removing them with UNLINK.
  • Hot keys: why one key can overload one server, and a copy in each app server.
  • A TTL on everything cached, and moving to a new scheme without a flag day.

More in Redis in production

Build "Designing the keyspace" yourself

GullyXI, a Pune fantasy cricket app, falls over on match night. You make it fast with Redis. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.

The first 2 chapters of Redis are free. This one comes with the all labs plan.