Building Relay

0.1 · From app to infrastructure — finding the real product

You will produce: A positioning statement; a non-goals list

0.2 · Four people who will judge us

You will produce: A persona set including the invisible end user

0.3 · Journeys — where products die

You will produce: Journey maps; the ★ moments

0.4 · Requirements you can test

You will produce: An SRS slice with IDs, priorities, verification methods

0.5 · Deciding out loud — the SAD and the ADR habit

You will produce: A drivers table; two ADRs written from scratch

1.1 · The monorepo and the toolchain

You will produce: A running pnpm workspace — TypeScript, lint, and a passing test suite

1.2 · One command, whole world

You will produce: A one-command local infrastructure — four stores, healthchecked and verified

1.3 · The protocol package

You will produce: The shared wire contract — frame types, error codes, and schemas that reject bad input

1.4 · Walking skeleton

You will produce: Two running skeleton services — health-checked, request-ID'd, logging structured JSON

2.1 · Schema with a spine

You will produce: A migrated schema and a tenant-scoped repository layer — cross-tenant leaks made inexpressible

2.2 · The write path

You will produce: POST message: channel row lock, sequence assignment (ADR-03)

2.3 · Send it twice

You will produce: Idempotency keys, partial unique index (DR-03)

2.4 · History that pages

You will produce: Cursor pagination on (channel_id, seq)

2.5 · The socket

You will produce: Gateway: WS termination, JWT verify, connection registry

2.6 · Two servers, one conversation

You will produce: Redis fan-out (ADR-07); the lossy-fabric argument

2.7 · The tunnel

You will produce: Resume protocol: cursors, backfill, subscribe-before-backfill buffer

2.8 · Milestone: the Tuan test

You will produce: An integration suite scripting journey 4 end-to-end — the SRS Phase 1 exit criterion

3.1 · Tenants all the way down

You will produce: Orgs, apps, environments; OAuth signup; the auto-created dev environment

3.2 · Keys and tokens — two credentials, one mistake

You will produce: API keys (prefix, hash, rotation); user JWTs; the dev-token endpoint

3.3 · Errors that resolve

You will produce: An error registry that declares every code the platform can send, one place that builds the docs_url, and a typed thrower so a mistyped code stops compiling

3.4 · The isolation harness

You will produce: A cross-tenant suite whose target list derives itself from the running router, three attack shapes that compare a pair of responses rather than a status code, a structural check that every table has a path back to one tenant, and the socket attacked from the protocol's own frame union

3.5 · The outbox

You will produce: Transactional outbox + relay (ADR-06); the crash-in-the-gap test

3.6 · JetStream and the first consumer

You will produce: Stream config; shared subject grammar; a durable pull consumer that dedupes

3.7 · Commit and publish are two instants

You will produce: The resume duplicate closed: a high-water mark that outlives the buffer

3.8 · The endpoints and the instruments

You will produce: The two public endpoints Part 3 needed, idempotency enforced by a unique index rather than in memory, every validation error naming its field for the first time, the global-operation guard watching nine tables instead of five, and repository branch coverage answered with a number

3.9 · The channel a customer controls

You will produce: A private channel type that decides something on all four of its doors, member removal and roles, archiving that refuses a send without announcing the channel exists, and a gauntlet that attacks your own tenant

3.10 · What a user sees

You will produce: Channel listing with cursor pagination and activity ordering, unread counts from the sequence the write path already maintains, user profiles created implicitly on first authentication, a deleted user whose messages survive, and a ban enforced at the door and on the send path

3.11 · The sender a message never had

You will produce: Bot users carrying a description the database requires, a sender on every message enforced by the compiler rather than a test, a credential that speaks as software and never as a person, refusals that reveal nothing about who exists, and a bot billed as active yet exempt from the send ceiling

3.12 · The message that never arrived

You will produce: A message sent over REST that reaches a live socket, an ordering that splits by transport because a request handler's response IS its acknowledgement, a publisher that survives a dead broker in 2 ms, and a P1 clause measured as unmet and recorded rather than narrowed

3.13 · Presence, and who is allowed to see it

You will produce: A frame in the protocol union since chapter 1.3 that finally has a producer, a second subject grammar leaving the message hot path byte-identical, a key whose existence is the state and whose SET … NX is the election, a grace period whose first fix stranded users online, and no filtering code

3.14 · The membership that changed under a live socket

You will produce: A clause unmet since chapter 2.6, closed by inverting the test that caught it; a third subject grammar addressing a principal rather than a channel, because an addition cannot ride a subject its instance never subscribed to; and a ban that arrives as one change and leaves as one frame per channel

3.15 · The frame nobody may send

You will produce: The first second inbound frame in twenty chapters, behind a named set a test pins; a fourth subject grammar taken rather than avoided, after re-deriving ADR-19's count and finding seven typed points where the record says three; and an honest verdict on a clause this platform cannot perform

3.16 · The sixth connection, and where the count lives

You will produce: FR-RTM-09 closed in both halves: a five-connection cap no gateway instance can compute alone, held as five slot keys claimed with `SET NX PX`; a sixth close code, the only refusal in the set whose correct handling is not a retry; and a cap that fails open loudly, the log line its only evidence

3.17 · The words somebody wants back

You will produce: FR-MSG-07, FR-MSG-08 and FR-MSG-10 built, and FR-RTM-05's last two event kinds given their first producers: a fifth subject grammar carrying both mutations with the kind in the payload, because a tombstone is not a `Message` and an edit is; and two error codes rather than the generic 403

3.18 · The message that is not only text

You will produce: FR-MSG-11's external-URL half built: a message carries attachments, bounded at ten and 2,048 characters, refused unless the scheme is http or https. The media_id half is deferred to §4.14 and refused by name, with a code of its own.

3.19 · Webhooks that survive the customer

You will produce: A dispatcher service: HMAC signing, a due-time retry schedule, dead letters

3.20 · When to stop trying

You will produce: Attempt records on an analytics stream, and auto-disable from two triggers

3.21 · The email nobody was sending

You will produce: The outbox pattern a third time, over a column the "When to stop trying" chapter already wrote — and Mailpit, because only a received message can prove an email carries no secret

3.22 · Limits you can see coming

You will produce: Per-environment request counters, the headers on every response, and two limiters that fail in opposite directions

3.23 · Quotas and what they cost

You will produce: Monthly quotas, spending caps, and degradation that rejects sends without touching history

3.24 · Counting a connection

You will produce: Connection-minutes metered from a service that owns no tables, a crash that under-bills by a bounded amount rather than over-billing for ever, and close code 4008 emitted for the first time since chapter 1.3 declared it

3.25 · Milestone: the isolation gauntlet

You will produce: Three deliberate reintroductions — one of which stayed green and taught the suite's range — and tests for the instruments that had never produced output

3.26 · Milestone: an outsider integrates

You will produce: A sealed integration package mechanically unable to import workspace code, and a verdict on the SRS Phase 2 exit criterion with what was measured and what was assumed

4.1 · The question the counters can't answer

You will produce: A million-message corpus, the analytical query written against Postgres for the first time, and four numbers showing the index that should fix it costs 49% more storage and buys noise

4.2 · The store that was never listening

You will produce: A ClickHouse a query can reach, an analytical schema with a ledger of its own, and the same question answered in 13.22 ms against Postgres's 585.9 — with one day out of ninety-one that can never reconcile

4.3 · The consumer that was promised

You will produce: The ingester for a stream that has been filling since chapter 3.20 with nothing reading it — deduplicating on the record's own key, because the template built to prevent exactly this problem writes to the database this path may not touch

4.4 · The requests that belong to nobody

You will produce: An analytical record for every API request, and the measurement that FR-ANL-01's "every request" and FR-ANL-07's "per tenant" are not the same population — 34 of 54 requests resolve to no tenant, and the gap is widest exactly where the traffic is

4.5 · The gateway's first stream

You will produce: FR-ANL-01's last arm without a producer, built in the one service that had never held a broker client — and the measurement that neither a clean stop nor a kill balances the opens against the closes, with the two connection counters wrong in opposite directions

4.6 · The rollup nobody read

You will produce: The rollup DR-10 asks for, after finding that the one already shipped is read by nothing and sits over a table nothing writes — and the measurement that put FR-ANL-09's channel dimension and DR-10's cheap read into separate tables, at 525x the rows and a billing query that touched more of them than the raw events it replaced

4.7 · The job that checks the meter

You will produce: FR-ANL-06's reconciliation job, and the finding that it cannot pass: two operational counters of the same quantity disagree by 0.2630% while 49 of 1,385 tenant-periods breach the bound, and three of the four quantities cannot meet 0.1% for reasons that are not defects — uniq's cliff at 65,536, a retention boundary that moves with the clock, and a connection ledger that counts a different population on each side

4.8 · The log a customer can search

You will produce: FR-ANL-07's query surface over the request log chapter 4.4 produced — tenant-scoped, paged on a composite cursor, filtered against the router's own route set — and the finding that the clause beside it cannot be built: FR-ANL-10 names a quantity this platform has never defined, over a column with 0 rows and 0 producers, and the function everyone would reach for reads the p99 at half its true value on the only real latency sample there is

4.9 · Milestone: the meter agrees

You will produce: An integration gate whose colour can change — 54 of 54 suites executing and reported, where a planted drift used to deepen a run that had been red since chapter 4.4 rather than turn it — and FR-ANL-06's agreement measured at 121,057 messages in one tenant-period, the first volume in this project where 0.1% is a threshold a whole number can sit either side of. Plus what the clause still lacks: no daily job runs anywhere, and the alert has no mechanism

4.10 · The upload that never reaches us

You will produce: An upload slot: one route that signs a fifteen-minute URL in twenty-eight lines of node:crypto, adding no dependency, so a client uploads a photo straight to object storage and the api's own request log shows the slot request and nothing else. Four refusals with four codes, only one of them transient. A storage quota enforced against a sum over the tenant's rows and serialised per tenant, with FR-RTL-05 amended to say why stored bytes are a level and not a monthly flow

4.11 · The half of the union that was refused

You will produce: The `{ type: "media" }` arm accepting, at all three doors that parse it, with one predicate inside the send's own transaction: the object belongs to this tenant, and for a user token it was uploaded by the sender or by the tenant itself. Three conditions get one answer, because naming which one failed would tell a caller whether somebody else's object exists. Plus the five forwarding readers that had to learn the new arm before the producer shipped — one of them drops a committed message with nothing but a log line — and the database's own refusal published as the evidence that FR-MED-06's `ready` state cannot be reached yet

4.12 · A link that expires, and who may hold it

You will produce: `GET /v1/media/:mediaId`, which answers a signed one-hour URL to a caller who may read a message referencing the object and the same 404 to everybody else. The authorisation rule is the platform's own `channelVisibleTo` rather than the membership check the clause's words describe — a literal reading would refuse a user the photo in a message whose text they can read. Plus the index that does not help the query it was built for, because the joined form the analysis passes produced cannot use it; the three tenancy predicates no single-mutation probe can see; and the caller-triggered 500 on sixteen shipped routes that this route is the first to escape

4.13 · The only service that reads the bytes

You will produce: A fifth service that sweeps for unverified objects, streams each one through ClamAV, checks the declared size against the store's own count and the declared type against the bytes, and posts a verdict back over the internal seam. The event the architecture document says it consumes has no producer and cannot have one, because the client uploads straight to the store — so the sweep the specification rejected as wasteful turns out to cost 4.2 seconds for the whole backlog, and the clause it satisfies cannot be made contingent on a client choosing to send a notice. Plus ADR-14's delivery gate, which turns ten tests red if it ships a chapter early; a health check that reads the scanner's signature-database date, because a liveness probe passes against one thirteen days old; and three test fixtures that were true statements about the platform until it grew something that reads the bytes

4.14 · Pending, ready, rejected

You will produce: The state machine's other half: every door that serves a message now serves the state its media object is in right now, read when the message is served rather than stored on it, and a frame goes out to every channel referencing the object when a verdict lands. The clause is two sentences and neither was met — the event everybody quotes, and a first sentence nobody had noticed was also unmet, because one schema was serving both the door a sender writes to and the payload the platform builds. Plus the subject question a review left open two parts ago, answered by arithmetic somebody had already written down; the cheapest shape available, killed by a field that does not exist; and four tests that asserted the absence this chapter fills, under a comment saying it would last until movement VI

4.15 · What a thumbnail costs

You will produce: A thumbnail for every uploaded image, and the answer to what one costs — which is not the number anybody reaches for first. Across a real corpus the ratio to the parent spans 775x while the thumbnail's own size spans 4x, so the ratio is a fact about the parent and publishing it as the cost publishes the wrong variable. Below the bound a thumbnail is 97.3% of the parent and the same pixels, which decides a behaviour rather than a figure. Plus the clause's last five words, which turn out to be the hard part: a derived object is named by no message, so one shipped clause refuses to serve it and another reaps it after a day, both working exactly as written — and nothing in the schema could say that two rows were related. The dependency that had to be argued for without borrowing the previous chapter's argument, the fourth round trip nothing had needed before, and a transaction the plan said was there

4.16 · Storage on the bill

You will produce: A tenant's stored bytes, metered daily and counted by kind, and the weekly comparison the clause asks for against what the object store actually holds. The comparison is the chapter: run against real data it says the meter charges 4,436 MB while the bucket holds 30.7 MB, and 99.9% of the difference is slots reserved and never uploaded to — a reservation measured against a delivery, which is FR-MED-01 working as designed rather than a drift. So the verdict has a third outcome, and the term that explains it cannot be computed from either store alone. Plus a column whose name reads as this chapter's subject and counts something else entirely, a materialised view that was counting zeros under a key it had no business creating, and a coverage number that was right about a service running in the wrong process

4.17 · Milestone: an image, end to end

You will produce: One image carried from an upload slot to delivered bytes by the software a deployment runs, with the verdict made by a container the test did not start — the join seven chapters had each built a piece of and nothing had ever checked. Plus a second image the worker refuses, arriving as a marker a reader can tell apart from a deleted message in two fields rather than one. The chapter is also an argument about what a milestone is for: its own premise turned out to be partly wrong, which made it smaller rather than larger, and the thing it found on the way was a comment explaining why an assertion was stable that was false in the job that runs it. What the path costs, with its sample size: every step a client controls is under 20 ms and the one it waits on is three orders of magnitude larger, uniform over a five-second sweep — and one upload in six waits five or six passes for a reason that is neither the timer nor the scanner, published rather than smoothed away

4.18 · The log that cannot be edited

You will produce: A table nothing in the application can change, and the decision that makes it worth having. FR-MOD-03 asks for an immutable audit log of every moderation action, and the clause names a population without saying who is in it — so the chapter's product is a classification of all 24 mutating routes a tenant can reach, each with a reason, checked in both directions against the routes a booted application reports. The set comes out at eight where a reader would predict nine, and the rule that produced it is wrong about two routes in the same direction, which is how the line it actually draws gets named: standing, not data. The word `immutable` costs a mechanism this schema had never used — the obvious one, revoking the privilege, does nothing at all against a superuser, and a unit test had forbidden the one that works for a reason that is the exact opposite of this case. Both ways around the trigger are measured and published beside the claim, because a security sentence with no attack against it is a comment

4.19 · Everything, including what was deleted

You will produce: The text a message held at the moment somebody removed it — which this platform threw away, every time, for twenty-five chapters. FR-MOD-01 asks for a channel's complete history including tombstones and edit history, and both of those nouns were already built: the chapter opens by running the premise and finding four of the clause's five obligations met. The hole is at the join. An edit records the text it REPLACED and a deletion recorded nothing, so a message edited twice and then deleted gives back two of its three texts and a message deleted with no edits gives back none of its one, and nothing anywhere notices. You will build the row that closes it, make the table append-only because FR-MSG-07 has said `immutable` since chapter 3.23 and nothing enforced it, and publish the boundary as two numbers rather than one — because a tombstone that kept its earlier texts looks served and is missing the only text a dispute turns on. Two findings cost more than the feature did: a microsecond column that has only ever held milliseconds, and a required field that reached a strict schema one seam away

4.20 · The messages that expire

You will produce: A retention policy a customer sets, a sweep that enforces it, and the two sentences the platform refuses to write. FR-MOD-06 asks for configurable retention with expired messages hard-deleted by a scheduled job, and the premise check found NONE of its three obligations met — the inverse of the chapter before it. The column had existed since chapter 2.1 and was set on 0 of 33,051 environments; nothing read it; and the hard deletion the clause names was refused by this platform's own schema on 5,495 messages. You will meet that refusal as a pincer: the foreign key stops the parent, the append-only trigger the previous chapter built stops the children, and `ON DELETE CASCADE` is refused as well — because a cascade issues an ordinary DELETE and a row trigger fires on it, which is the measurement that decides the whole design. The only thing that works unchanged is the bypass the previous chapter published as the limit of its own guarantee. So the products are a narrow auditable exception and a new ADR, and the ADR's first decision is a reading rather than a mechanism: three documents reserve hard deletion for the compliance path and the constitution's own word is PATH where the SRS said ENDPOINT, so the rule hardest to change is the one that already permitted this. What the chapter cannot do is run the sweep: there is no scheduler, this is the fourth clause bounded by its absence, and it is the first one where the absence is a customer telling an auditor that data does not exist

4.21 · Erasure, and every path it must find

You will produce: An endpoint that erases a person from seven stores, and a receipt that names the one it cannot. FR-MOD-04 asks for permanent erasure of messages, memberships, profile and analytical records — and the chapter opens on a behaviour rather than an error: you delete a user the way the platform already lets you, and find the row, the messages and the billing rows still there, all of it correct. The hard part is the word MESSAGES. FR-USR-05 preserves them and FR-MOD-04 destroys them, both clauses are right, and the reading that resolves it is the one Slack and Microsoft Teams both take: a compliance erasure asks for removal of a person, not of a conversation. Everything else follows from that single decision. Keeping the messages means the author's row can never be deleted, because all five foreign keys refuse it — so erasure leaves a tombstone, and a tombstone is what makes every store that references the user BY KEY stop naming anybody without being touched. That is a new ADR, and it turns what was going to be the chapter's weakest limitation into its argument. You will also watch a tenant-scoped delete statement take 1,113 of 1,116 rows belonging to other tenants, because one value came out of a URL path and went into SQL unbound — and you will see the published payload for that attack fail to reproduce it, which is its own lesson about probes. The store that cannot comply is not an analytical one: it is the audit log, which holds the person's external id on every row naming them and is append-only on purpose. The erasure writes one more. The one place the name survives is the record that the name was erased

4.22 · The identifier the customer gave it

You will produce: Thirteen routes that take the name your customer already had for a channel, and the one query that makes them. FR-USR-01 says Relay shall not generate end-user identities and ADR-18 says an end user's identity is whatever external_id the customer already had — and yet every route beneath /v1/channels/:channelId wanted a uuid Relay minted, so a customer had to keep a lookup table mapping their order number to our key. The exact table the journey map promises they will not need. You open on the failure: create a channel called order-88412, ask for it back by that name, and get a 500. Not a 404 — an internal error, which tells a developer the platform is broken rather than that they used the wrong key. The cause is three layers down and it is a cast: 'order-88412'::uuid raises in Postgres before the OR beside it can short-circuit, which is simultaneously why the error happens and why one query cannot resolve both forms. So the fix and the repair are a single edit, and the chapter's first product is a reading rather than a mechanism. You will build the resolution as a pipe and find out why the cheaper design cannot work: middleware runs BEFORE guards, so it has no principal, and a resolution with no environment is a cross-tenant read. Then you will write the pipe wrong in the way that looks right — throwing the same 404 every handler throws — and watch the isolation gauntlet catch it, because a pipe answers before every check the handler makes, and a banned user could suddenly tell a real channel from an invented one. You will measure what the resolution costs rather than assert it is cheap, delete the tenancy scope to see which tests notice and find that the suite built for that question notices nothing, and sweep every response the API returns for an identifier no route accepts — which turns up one, on a write route everyone uses, that a published ADR said could not exist

4.23 · The channel a socket names

You will produce: A real-time surface that calls a channel what your customer calls it, and a lesson in counting the right things. Chapter 4.22 made thirteen REST routes take the identifier the customer gave a channel; the gateway did not change and was never asked to, so one platform answered two ways and Journey 3's “zero lookup tables” held on the surface a support tool asks questions with and not on the surface it watches. You open on that: the same channel, named order-88412 over REST and 419320ca-… on the socket, one minute apart. Then you size the work and get it wrong twice. Seven schemas carry a channel field — and connection.ack names channels three more times WITHOUT the word, in two maps keyed by channel and a list of them, so a grep for a field cannot see a map keyed by that field's value. Twenty-one gateway expressions write a channel — and only three build a client frame; eleven are log lines an operator reads against the subject names, six are internal publishes, and the commonest frame on the socket is forwarded from the api and written by no gateway expression at all. So the rename goes at the single send, and that turns out to be a correctness choice rather than a tidy one: the resume buffer holds frames that two internal comparisons index by channel, so translating where a frame is BUILT re-sends a resuming client its whole backlog and flushes a revoked channel's messages anyway, both silently. You will build the map in both directions, find the hole where a channel joined mid-session cannot be named, discover that the sixty-second backstop cannot cover it, and then watch the lane catch a defect the reading missed — the backstop's own additions carried no identity and the new send dropped them. You will measure 19 of 44,574 identifiers that are another channel's key, which is why no shape test can separate the two forms in a resume cursor, and you will write three cross-tenant attacks by hand because a suite that derives its targets from the protocol is green by construction against a change in what a field carries