Part 3 · Chapter 3.4
You will produce: A cross-tenant suite whose target list derives itself from the running router and fails on a route it has no decision about, three attack shapes that compare a pair rather than a status code, a structural check that every table has a path back to one tenant, and the socket surface attacked from the protocol's own frame union · about 55 minutes including the exercise
Source: SRS — Software Requirements Specification · SAD — Software Architecture Document
Constitution I says one tenant's data must never reach another. Every chapter so far has
agreed with it, and a few have tested it — messages.itest.ts checks that a foreign
channel is not found, and it is right to.
The trouble with nine scattered assertions is not that any of them is wrong. It is that none of them can tell you what is missing. A route added next week with no isolation test looks exactly like a route that has one, because the thing you would be looking for is an absence.
So this chapter builds a suite that cannot have that problem: one that derives what to attack from the running application, and fails when it finds something it has no decision about.
The fault worth preventing is a route that exists and is unattacked. Only the router knows what exists, so the router is what gets asked:
/** The gauntlet's target list (NFR-SEC-09).
*
* A LIST OF CLASSIFICATIONS, NOT A LIST OF TARGETS. The targets themselves are derived
* from the running application — `app.getHttpAdapter().getInstance().router.stack` —
* because the fault this suite exists to prevent is a route that exists and is
* unattacked, and only the router knows what exists. What lives here is the decision
* about each one, which no derivation can make.
*
* NOTHING MAY BE EXEMPT BY OMISSION. A derived target matching no entry fails the
* suite, and an entry matching no derived target fails it too — the second direction is
* the one that catches a stale exemption after a rename. That pair of assertions is the
* whole mechanism. */
/** What kind of attack a route takes.
*
* `credential` is the shape a foreign-identifier attack cannot express. `POST
* /auth/dev-token` accepts no tenant-owned identifier, so there is nothing to put in
* one — and it is tenant-scoped all the same, because the key it accepts resolves to
* exactly one environment. Filing it as `exempt` is how a route stops being attacked
* while looking accounted for. */
export type Shape = "read" | "write" | "credential" | "exempt";
/** Which credential class the route accepts, and therefore which attack applies.
*
* A `write` shape alone cannot tell these apart. A route taking an end-user token is
* already scoped to one environment, so the attack is a FOREIGN CREDENTIAL. A route
* taking an application key is scoped too, but by a different resolution — and a route
* that accepts either is attacked as both, which is why `either` is a class rather
* than a shrug. */
export type CredentialClass = "application" | "user" | "either" | "none";
interface Classified {
method: string;
path: string;
accepts: CredentialClass;
}
/** `exempt` requires a reason. The type is what makes that true — an exemption with no
* argument beside it is indistinguishable from an oversight six months later. */
export type Classification =
| (Classified & { shape: Exclude<Shape, "exempt">; because?: string })
| (Classified & { shape: "exempt"; because: string });
/** Every route the api serves, as the derivation reports it: 9 today.
*
* The shapes sum to 9 — 3 exempt, 2 credential, 1 read, 3 write — and that sum is
* asserted against the derivation rather than written down twice. A count nobody
* recomputes is a count that stops being true quietly.
*
* THIS LIST GROWS WITH THE PLATFORM. Every chapter after this one that adds a route
* adds a row here, and the suite goes red until it does. That is the point of building
* the gauntlet now rather than at the end: a target list written after the fact is a
* list somebody reconstructs from memory, and the routes it forgets are exactly the
* ones nobody was thinking about. */
export const CLASSIFICATIONS: readonly Classification[] = [
// ── exempt: no tenant-owned identifier and no tenant-scoped credential ──────────
{
method: "GET",
path: "/healthz",
accepts: "none",
shape: "exempt",
because:
"liveness only. No credential, no identifier, and the body is a service name and an uptime.",
},
{
method: "GET",
path: "/auth/:provider/start",
accepts: "none",
shape: "exempt",
because:
"pre-tenant. `:provider` names an identity provider, not anything a tenant owns, and the caller has no credential yet — this is the route that gets them one.",
},
{
method: "GET",
path: "/auth/:provider/callback",
accepts: "none",
shape: "exempt",
because:
"pre-tenant, and the route that CREATES the tenant. There is no second tenant to reach across from, because at this point the caller belongs to none.",
},
// ── credential: tenant-scoped, but with nothing to put a foreign id into ────────
{
method: "POST",
path: "/auth/dev-token",
accepts: "application",
shape: "credential",
because:
"the body names no tenant-owned resource, so a foreign-identifier attack has nothing to express. It is tenant-scoped all the same: the key it accepts resolves to exactly one environment, and the token it mints must not outlive that scope.",
},
// ── the public message surface: attacked as both classes, because it takes both ─
{
method: "POST",
path: "/v1/channels/:channelId/messages",
accepts: "either",
shape: "write",
},
{
method: "GET",
path: "/v1/channels/:channelId/messages",
accepts: "either",
shape: "read",
},
// ── the internal surface: an end-user token, so a FOREIGN CREDENTIAL is the attack
{ method: "POST", path: "/internal/messages", accepts: "user", shape: "write" },
{ method: "POST", path: "/internal/backfill", accepts: "user", shape: "write" },
{
// NOT A `write`, AND THE DIFFERENCE IS THE WHOLE POINT OF HAVING SHAPES. This route
// takes no body and no path parameter: there is no identifier to forge, so a
// foreign-identifier attack has nothing to express. Its only tenant-scoped input is
// the token, which is what `credential` attacks. Classifying it `write` would have
// produced an attack that sends a valid request and proves nothing.
method: "POST",
path: "/internal/session",
accepts: "user",
shape: "credential",
},
];
export function targetKey(t: { method: string; path: string }): string {
return `${t.method.toUpperCase()} ${t.path}`;
}
/** Counts, for the suite to print. Derived from the list rather than typed beside it,
* because a hand-maintained tally is the thing that goes stale first. */
export function shapeCounts(list: readonly Classification[]): Record<Shape, number> {
// NO `list` SHAPE YET, and that is deliberate rather than an omission. Nothing this
// api serves returns a collection, so a list attack would be a function with no
// target — and a shape with no member is a vocabulary entry that drifts. The chapter
// that adds the first list route adds the shape and the attack together.
const counts: Record<Shape, number> = { read: 0, write: 0, credential: 0, exempt: 0 };
for (const c of list) counts[c.shape]++;
return counts;
}
/** One routable endpoint, as the running application reports it. */
export interface DerivedTarget {
method: string;
path: string;
}
/** The shape of the express router this reaches into. Declared rather than imported:
* express 5 ships no types, and adding `@types/express` for one property read would
* move the api's dependency list for a test's benefit. */
interface RouterLike {
stack?: Array<{
route?: { path?: string; methods?: Record<string, boolean> };
name?: string;
}>;
}
interface AdapterInstance {
router?: RouterLike;
_router?: RouterLike;
}
/** Where the router lives, and the fact that this is not public API.
*
* Express 5 exposes `router`; express 4 called it `_router`. Both are read, and WHICH
* ONE ANSWERED is returned so a test can assert the derivation found something rather
* than silently finding nothing. That is the failure mode worth designing against: a
* renamed property would leave this suite green while it attacked zero routes, which is
* worse than the hand-written list it replaces. */
export function deriveTargets(instance: unknown): {
targets: DerivedTarget[];
middlewareLayers: number;
property: "router" | "_router" | "none";
} {
const adapter = instance as AdapterInstance;
const router = adapter.router ?? adapter._router;
const property = adapter.router ? "router" : adapter._router ? "_router" : "none";
const targets: DerivedTarget[] = [];
let middlewareLayers = 0;
for (const layer of router?.stack ?? []) {
if (!layer.route) {
middlewareLayers++;
continue;
}
const path = layer.route.path ?? "";
for (const [verb, on] of Object.entries(layer.route.methods ?? {})) {
if (on) targets.push({ method: verb.toUpperCase(), path });
}
}
return { targets, middlewareLayers, property };
}
/** A route that has existed since chapter 2.2 and will exist for as long as this
* product does. If the derivation cannot find THIS, it has not found the mounted
* router, whatever else it returned. */
export const CANARY_TARGET = "POST /v1/channels/:channelId/messages";What lives in that file is not a list of targets. It is a list of decisions — one per
route, saying which attack applies and, for anything exempt, why. The targets themselves
come off router.stack at run time.
That is why the derivation reports which property answered, and why the first three tests are about the derivation rather than about isolation:
import "reflect-metadata";
import type { INestApplication } from "@nestjs/common";
import { Test } from "@nestjs/testing";
import { afterAll, beforeAll, describe, expect, it } from "vitest";
import { AppModule } from "../app.module";
import {
CANARY_TARGET,
CLASSIFICATIONS,
deriveTargets,
shapeCounts,
targetKey,
type DerivedTarget,
} from "./targets";
// THE DERIVATION'S OWN TESTS, and they matter more than they look.
//
// The gauntlet attacks a list it derives from the running application rather than one
// somebody typed, because the fault NFR-SEC-09 exists to prevent is a route that exists
// and is unattacked. But a derivation has a failure mode a typed list does not: IT CAN
// RETURN NOTHING AND PASS. Express 5 renamed `_router` to `router`; an upgrade that
// renamed it again would leave this suite green while it attacked zero endpoints, which
// is worse than the hand-written list it replaces, because a hand-written list at least
// looks wrong when you read it.
//
// So the derivation is checked before it is used: it found a router, it found routes, it
// found the route we know is there, and every target it found is classified exactly once.
describe("the gauntlet's target list derives from the running application", () => {
let app: INestApplication;
let derived: DerivedTarget[];
let middlewareLayers: number;
let property: string;
beforeAll(async () => {
const moduleRef = await Test.createTestingModule({ imports: [AppModule] }).compile();
app = moduleRef.createNestApplication();
await app.init();
const result = deriveTargets(app.getHttpAdapter().getInstance());
derived = result.targets;
middlewareLayers = result.middlewareLayers;
property = result.property;
}, 60_000);
afterAll(async () => {
await app?.close();
});
it("found a router at all, and says which property answered", () => {
expect(property).not.toBe("none");
expect(["router", "_router"]).toContain(property);
});
it("found routes — an empty list is a broken derivation, not a clean surface", () => {
expect(derived.length).toBeGreaterThan(0);
// Middleware layers exist in any mounted express app. Zero of them alongside zero
// routes is the signature of reading the wrong object rather than of a small api.
expect(middlewareLayers).toBeGreaterThan(0);
});
it("found the route that has existed since chapter 2.2", () => {
expect(derived.map(targetKey)).toContain(CANARY_TARGET);
});
it("classifies every derived target exactly once", () => {
const entries = new Map<string, number>();
for (const c of CLASSIFICATIONS) {
entries.set(targetKey(c), (entries.get(targetKey(c)) ?? 0) + 1);
}
const unclassified = derived.map(targetKey).filter((k) => !entries.has(k));
const duplicated = [...entries].filter(([, n]) => n > 1).map(([k]) => k);
// Named in the failure, because "9 !== 10" sends a reader to count rows.
expect({ unclassified, duplicated }).toEqual({ unclassified: [], duplicated: [] });
});
it("has no classification entry that matches no derived target", () => {
const found = new Set(derived.map(targetKey));
const stale = CLASSIFICATIONS.map(targetKey).filter((k) => !found.has(k));
// THIS IS THE DIRECTION THAT CATCHES A RENAME. A route renamed with its exemption
// left behind is a route nobody attacks and nobody misses.
expect(stale).toEqual([]);
});
it("gives every exempt entry a reason", () => {
const reasonless = CLASSIFICATIONS.filter(
(c) => c.shape === "exempt" && (!c.because || c.because.trim() === ""),
).map(targetKey);
expect(reasonless).toEqual([]);
});
it("accounts for every derived target as attacked or exempt", () => {
const counts = shapeCounts(CLASSIFICATIONS);
const attacked = counts.read + counts.write + counts.credential;
// A number nobody can see is a number nobody checks. Visible under
// `--reporter=verbose`; the assertion below is what gates the build either way.
console.log(
`gauntlet targets: ${derived.length} derived, ${attacked} attacked, ${counts.exempt} exempt ` +
`(read ${counts.read}, write ${counts.write}, credential ${counts.credential})`,
);
expect(attacked + counts.exempt).toBe(derived.length);
});
});Run it and it says what it found:
gauntlet targets: 9 derived, 6 attacked, 3 exempt (read 1, write 3, credential 2)Nine routes today. Every chapter after this one that adds a route adds a row to that list, and the suite goes red until it does. That is the point of building the gauntlet now rather than at the end of the part: a target list written afterwards is a list somebody reconstructs from memory, and the routes it forgets are exactly the ones nobody was thinking about.
An attack needs a victim. One environment can only prove that a made-up uuid is not found, which is a much weaker claim than the one the constitution makes:
import { createApiKey, createEnvironment, Repository } from "../db/repository";
import type { Db } from "../db/client";
/** Two tenants, so every attack has a victim and an attacker.
*
* The gauntlet's unit of assertion is a PAIR of requests — another tenant's identifier
* and an identifier that exists nowhere — and it needs a second tenant to borrow
* identifiers from. One environment can only prove that a made-up uuid is not found,
* which is a much weaker claim than the one constitution I makes.
*
* EVERY ROW IS SCOPED TO AN ENVIRONMENT THIS FUNCTION MINTED, AND NOTHING HERE DELETES
* ANYTHING. A fixture that tidied up with a `DELETE FROM channels` would be a global
* operation asserting a local fact, and the lane shares one database — that is the
* trade chapter 2.1 made when it put `environment_id` in every table, and it pays for
* itself here. The environments are disposable; leaving them costs rows and no
* correctness.
*
* ONE OF EACH KIND OF ROW THE ROUTES TOUCH, because the write attacks read storage
* before and after: a channel and a message for the message routes, a user for the
* session route. */
export interface Tenant {
environmentId: string;
/** An `rk_dev_…` credential for this environment, minted the way signup does. */
credential: string;
userId: string;
userExternalId: string;
channelId: string;
/** The customer-supplied identifier, so an attack can present the other tenant's own
* external id rather than only its uuid. */
channelExternalId: string;
messageId: string;
repo: Repository;
}
export interface TwoTenants {
/** The caller. Its credential is the one every attack presents. */
attacker: Tenant;
/** The tenant whose identifiers the attacker borrows. Nothing it owns may move. */
victim: Tenant;
}
async function seedTenant(db: Db, label: string): Promise<Tenant> {
const environment = await createEnvironment(db, { name: `isolation-${label}` });
const key = await createApiKey(db, { environmentId: environment.id });
const repo = new Repository(db, environment.id);
const userExternalId = `${label}-user`;
const user = await repo.createUser(userExternalId, `${label} user`);
const channelExternalId = `${label}-channel`;
const channel = await repo.createChannel(channelExternalId, "public", label);
await repo.addMember(channel.id, user.id);
const message = await repo.sendMessage(channel.id, {
text: `${label} says something`,
userId: user.id,
});
return {
environmentId: environment.id,
credential: key.credential,
userId: user.id,
userExternalId,
channelId: channel.id,
channelExternalId,
messageId: message.id,
repo,
};
}
/** Seeded sequentially rather than with `Promise.all`, so a failure names which tenant
* failed to seed instead of rejecting whichever lost the race. */
export async function seedTwoTenants(db: Db): Promise<TwoTenants> {
const attacker = await seedTenant(db, `attacker-${Date.now().toString(36)}`);
const victim = await seedTenant(db, `victim-${Date.now().toString(36)}`);
return { attacker, victim };
}Nothing in there deletes anything, and that is deliberate. A fixture that tidied up with
a DELETE FROM channels would be making a global change to assert a local fact — the
lane shares one database, and every other suite's rows are in it. Leaving the
environments behind costs rows and no correctness, which is the trade chapter 2.1 made
when it put environment_id in every table.
Here is the design decision the whole suite turns on.
The obvious assertion is expect(res.status).toBe(404). It is wrong in three ways at
once. It freezes today's status choices into a security test, so a considered change to a
status breaks the suite for no security reason. It says nothing about the body. And it
passes an endpoint that leaks through its prose.
/** The three attacks, one per shape (NFR-SEC-09).
*
* THE UNIT OF ASSERTION IS A PAIR, NOT A REQUEST. Constitution I forbids revealing
* that another tenant's data exists, so the correct answer to a foreign identifier is
* whatever the platform says about an identifier that exists NOWHERE — and a single
* response cannot show that. Every attack here issues both and compares them.
*
* A suite asserting `404` instead would be wrong in three ways at once. It would
* freeze today's status choices into a test, so a considered change to a status breaks
* a security suite for no security reason. It would say nothing about the body. And it
* would PASS AN ENDPOINT THAT LEAKS THROUGH ITS PROSE — an error message that echoes
* the identifier back makes the foreign answer differ from the absent one while both
* are 404, which is exactly the leak constitution I is about.
*
* So status and whole body are compared. There is nothing to exclude from the
* comparison yet: the error envelope is `code`, `message` and `docs_url`, all three of
* which must match. When a per-request field joins it, the chapter that adds it owns
* the decision to drop it here — and it will have to argue that the field reveals
* nothing about the resource. */
export interface AttackRequest {
method: string;
/** Path with identifiers already substituted — `/v1/channels/<uuid>/messages`. */
path: string;
body?: unknown;
}
export interface Answer {
status: number;
body: unknown;
}
/** What an attack found. `differences` is empty when the pair is indistinguishable;
* when it is not, it says WHAT differed, because "expected true to be false" sends a
* reader back to the source and a named difference does not. */
export interface Verdict {
differences: string[];
foreign: Answer;
absent: Answer;
}
async function send(
baseUrl: string,
credential: string,
req: AttackRequest,
): Promise<Answer> {
const res = await fetch(`${baseUrl}${req.path}`, {
method: req.method,
headers: {
authorization: `Bearer ${credential}`,
...(req.body === undefined ? {} : { "content-type": "application/json" }),
},
...(req.body === undefined ? {} : { body: JSON.stringify(req.body) }),
});
const text = await res.text();
let body: unknown = text;
try {
body = text === "" ? null : JSON.parse(text);
} catch {
/* a non-JSON body is itself the answer, and comparing it verbatim is correct */
}
return { status: res.status, body };
}
/** Exported so a unit test can drive it. THE ARM THAT REPORTS A DIFFERENCE NEVER
* EXECUTES IN A HEALTHY LANE — every attack in the gauntlet compares equal — so the one
* branch that matters here is the one a passing suite cannot reach. Same shape as
* `classifyRow`: an instrument that has never fired is an untested instrument. */
export function comparePair(foreign: Answer, absent: Answer): string[] {
const differences: string[] = [];
if (foreign.status !== absent.status) {
differences.push(`status ${foreign.status} (foreign) vs ${absent.status} (absent)`);
}
const f = JSON.stringify(foreign.body);
const a = JSON.stringify(absent.body);
if (f !== a) differences.push(`body ${f} (foreign) vs ${a} (absent)`);
return differences;
}
/** A read of another tenant's resource must answer as a read of nothing. */
export async function readAttack(
baseUrl: string,
credential: string,
foreignReq: AttackRequest,
absentReq: AttackRequest,
): Promise<Verdict> {
const foreign = await send(baseUrl, credential, foreignReq);
const absent = await send(baseUrl, credential, absentReq);
return { differences: comparePair(foreign, absent), foreign, absent };
}
/** A write against another tenant's identifier must change nothing, and the pair must
* still be indistinguishable.
*
* THE STATE READ IS THE POINT. A 404 that COMPLETED the write is the case no status
* code reveals, and it is the one a reader should worry about. `readVictimState` is
* supplied by the caller and goes through `Repository` methods rather than raw SQL: the
* lint ban forbids the query engine outside `services/api/src/db`, and this suite
* should not need an exemption to do its job. */
export async function writeAttack(
baseUrl: string,
credential: string,
foreignReq: AttackRequest,
absentReq: AttackRequest,
readVictimState: () => Promise<unknown>,
): Promise<Verdict & { stateChanged: boolean; before: unknown; after: unknown }> {
const before = await readVictimState();
const foreign = await send(baseUrl, credential, foreignReq);
const absent = await send(baseUrl, credential, absentReq);
const after = await readVictimState();
return {
differences: comparePair(foreign, absent),
foreign,
absent,
stateChanged: JSON.stringify(before) !== JSON.stringify(after),
before,
after,
};
}
/** The shape a foreign-identifier attack cannot express.
*
* `POST /auth/dev-token` accepts no tenant-owned identifier, so there is nothing to
* forge — and it is tenant-scoped all the same, because the key it accepts resolves to
* exactly one environment. The attack is therefore ON THE CREDENTIAL: mint a token with
* environment A's key and present it where only B's users belong. Filing this route as
* exempt is how a route stops being attacked while looking accounted for. */
export interface CredentialVerdict {
minted: boolean;
/** The status the borrowed token got on the other tenant's resource. */
crossStatus: number;
crossBody: unknown;
}
export async function credentialAttack(
baseUrl: string,
attackerCredential: string,
user: string,
victimReq: AttackRequest,
): Promise<CredentialVerdict> {
const mint = await fetch(`${baseUrl}/auth/dev-token`, {
method: "POST",
headers: {
authorization: `Bearer ${attackerCredential}`,
"content-type": "application/json",
},
body: JSON.stringify({ user }),
});
if (!mint.ok) {
return { minted: false, crossStatus: mint.status, crossBody: await mint.json() };
}
const { token } = (await mint.json()) as { token: string };
const answer = await send(baseUrl, token, victimReq);
return { minted: true, crossStatus: answer.status, crossBody: answer.body };
}Three attacks, not four. There is no list route on this platform yet, so a list attack
would be a function with no target — the chapter that adds the first collection endpoint
adds the shape and its attack together. And there is nothing to exclude from the body
comparison: the envelope is code, message and docs_url, and all three must match.
When a per-request field joins it, the chapter that adds it owns the argument for
dropping it here.
The suite drives one attack per non-exempt route, and then checks that it did:
import "reflect-metadata";
import type { INestApplication } from "@nestjs/common";
import { Test } from "@nestjs/testing";
import { afterAll, beforeAll, describe, expect, it } from "vitest";
import { AppModule } from "../app.module";
import { createDb, createPool } from "../db/client";
import { credentialAttack, readAttack, writeAttack } from "./attack";
import { seedTwoTenants, type TwoTenants } from "./fixtures";
import { CLASSIFICATIONS, targetKey } from "./targets";
import type { Db } from "../db/client";
// THE GAUNTLET (NFR-SEC-09, constitution I).
//
// Two tenants over real HTTP. The attacker holds a valid credential for its own
// environment and presents the victim's identifiers with it. Every attack asserts a
// PAIR — the foreign identifier and one that exists nowhere must be indistinguishable —
// because "it returned 404" is a claim about a status code and constitution I is a
// claim about what a caller can learn.
//
// AN ATTACK PER NON-EXEMPT ROUTE, AND THE SUITE PROVES IT RAN THEM. A classification
// saying `write` with no attack written for it is the same hole as a route with no
// classification, one level up. The last test in this file compares what ran against
// what the list says should have run.
const ABSENT_UUID = "00000000-0000-4000-8000-000000000000";
describe("the isolation gauntlet", () => {
let app: INestApplication;
let url: string;
let db: Db;
let pool: ReturnType<typeof createPool>;
let t: TwoTenants;
/** A token minted with the ATTACKER's key, for the routes that take one. */
let attackerToken: string;
const attacked = new Set<string>();
beforeAll(async () => {
pool = createPool();
db = createDb(pool);
t = await seedTwoTenants(db);
app = (
await Test.createTestingModule({ imports: [AppModule] }).compile()
).createNestApplication({ logger: false });
await app.listen(0);
url = await app.getUrl();
const mint = await fetch(`${url}/auth/dev-token`, {
method: "POST",
headers: {
authorization: `Bearer ${t.attacker.credential}`,
"content-type": "application/json",
},
body: JSON.stringify({ user: t.attacker.userExternalId }),
});
expect(mint.ok, "the attacker must be able to mint a token for its OWN user").toBe(
true,
);
attackerToken = ((await mint.json()) as { token: string }).token;
}, 60_000);
afterAll(async () => {
await app?.close();
await pool?.end();
});
// ── read ────────────────────────────────────────────────────────────────────────
it("GET /v1/channels/:channelId/messages — a foreign channel reads as an absent one", async () => {
attacked.add("GET /v1/channels/:channelId/messages");
const verdict = await readAttack(
url,
t.attacker.credential,
{ method: "GET", path: `/v1/channels/${t.victim.channelId}/messages` },
{ method: "GET", path: `/v1/channels/${ABSENT_UUID}/messages` },
);
expect(verdict.differences, verdict.differences.join("; ")).toEqual([]);
});
// ── write ───────────────────────────────────────────────────────────────────────
it("POST /v1/channels/:channelId/messages — refuses, and writes nothing", async () => {
attacked.add("POST /v1/channels/:channelId/messages");
const verdict = await writeAttack(
url,
t.attacker.credential,
{
method: "POST",
path: `/v1/channels/${t.victim.channelId}/messages`,
body: { text: "from the attacker" },
},
{
method: "POST",
path: `/v1/channels/${ABSENT_UUID}/messages`,
body: { text: "from the attacker" },
},
() => t.victim.repo.listMessages(t.victim.channelId, { limit: 50 }),
);
expect(verdict.differences, verdict.differences.join("; ")).toEqual([]);
// THE STATE READ IS THE POINT: a 404 that completed the write is the case no
// status code reveals.
expect(verdict.stateChanged, "the victim's messages moved").toBe(false);
});
it("POST /internal/messages — a foreign channel_id refuses, and writes nothing", async () => {
attacked.add("POST /internal/messages");
const verdict = await writeAttack(
url,
attackerToken,
{
method: "POST",
path: "/internal/messages",
body: { channel_id: t.victim.channelId, text: "from the attacker" },
},
{
method: "POST",
path: "/internal/messages",
body: { channel_id: ABSENT_UUID, text: "from the attacker" },
},
() => t.victim.repo.listMessages(t.victim.channelId, { limit: 50 }),
);
expect(verdict.differences, verdict.differences.join("; ")).toEqual([]);
expect(verdict.stateChanged, "the victim's messages moved").toBe(false);
});
it("POST /internal/backfill — a foreign channel yields nothing", async () => {
attacked.add("POST /internal/backfill");
const verdict = await writeAttack(
url,
attackerToken,
{
method: "POST",
path: "/internal/backfill",
body: { cursors: { [t.victim.channelId]: 0 } },
},
{ method: "POST", path: "/internal/backfill", body: { cursors: { [ABSENT_UUID]: 0 } } },
() => t.victim.repo.listMessages(t.victim.channelId, { limit: 50 }),
);
expect(verdict.differences, verdict.differences.join("; ")).toEqual([]);
expect(verdict.stateChanged).toBe(false);
// AND THE BODY MUST CARRY NO ROWS. A backfill answers with messages rather than a
// status, so "indistinguishable from absent" is necessary and not sufficient — an
// endpoint that returned the victim's messages for BOTH requests would pass the
// pair comparison and fail the product.
const serialised = JSON.stringify(verdict.foreign.body ?? "");
expect(serialised).not.toContain(t.victim.messageId);
expect(serialised).not.toContain("victim says something");
});
// ── credential ──────────────────────────────────────────────────────────────────
it("POST /auth/dev-token — a token minted for one environment is refused by another", async () => {
attacked.add("POST /auth/dev-token");
const verdict = await credentialAttack(
url,
t.attacker.credential,
// The attacker asks for a token naming the VICTIM's user. Either the mint
// refuses, or the token it hands back must be useless against the victim.
t.victim.userExternalId,
{ method: "GET", path: `/v1/channels/${t.victim.channelId}/messages` },
);
if (verdict.minted) {
expect(
verdict.crossStatus,
`a token minted with the attacker's key reached the victim's channel: ` +
JSON.stringify(verdict.crossBody),
).not.toBe(200);
}
});
it("POST /internal/session — the attacker's token resolves only its own environment", async () => {
attacked.add("POST /internal/session");
const res = await fetch(`${url}/internal/session`, {
method: "POST",
headers: { authorization: `Bearer ${attackerToken}` },
});
const body: unknown = res.ok ? await res.json() : null;
const serialised = JSON.stringify(body ?? "");
// Whatever it answers, nothing of the victim's may appear in it.
expect(serialised).not.toContain(t.victim.userId);
expect(serialised).not.toContain(t.victim.channelId);
expect(serialised).not.toContain(t.victim.environmentId);
});
// ── and the suite accounts for itself ───────────────────────────────────────────
it("ran an attack for every route the classification says to attack", () => {
const shouldAttack = CLASSIFICATIONS.filter((c) => c.shape !== "exempt").map(targetKey);
const missing = shouldAttack.filter((k) => !attacked.has(k));
// A classification saying `write` with no attack written for it is the same hole
// as a route with no classification, one level up. Named, because the useful half
// of this failure is WHICH route nobody wrote a test for.
expect(missing, `classified but never attacked: ${missing.join(", ")}`).toEqual([]);
});
});That last test is the same idea one level up. A classification saying write with no
attack written for it is the same hole as a route with no classification.
The gauntlet attacks endpoints. There is a second question it cannot ask: a table with no path back to an environment is exposed by the first query that joins it, and nothing before that moment fails.
import { sql } from "drizzle-orm";
import type { Db } from "./client";
// WHERE EVERY TABLE KEEPS ITS TENANT (FR-012, constitution I).
//
// The gauntlet attacks endpoints. This asks the other half of the question: a table
// that carries no path back to an environment is A LEAK WITH NO ENDPOINT YET — nothing
// is exposing it today, and the first query that joins it will.
//
// DERIVED FROM `information_schema`, NOT FROM `schema.ts`, because the database is what
// the queries run against. A table added by a migration and never modelled in Drizzle is
// invisible to a check that reads the model, and that is exactly the table this exists
// to catch.
//
// It lives here rather than in the test that calls it because this directory is the only
// place the lint ban permits `drizzle-orm` (constitution I, ADR-16). A catalogue query
// written inline in the test would need an exemption for as long as it lived.
/** How a row in this table is traced back to one environment. */
export type TenantPath = "direct" | "hop" | "spine";
export interface TableClassification {
table: string;
/** `null` means the table matches none of the three, which fails the check. */
path: TenantPath | null;
/** For `hop`: the `direct` tables its foreign keys reach. */
via: string[];
/** For `spine`: why it has no tenant column. */
reason?: string;
}
// THE SPINE, AS A LIST WITH A REASON EACH AND NOT A PATTERN.
//
// A pattern silently absorbs the next table that happens to match it, which is the
// opposite of what this check is for. Adding a table here should be an edit somebody
// has to justify in writing, and the reason is stored so the justification outlives
// the person who made it.
//
// These six are tenancy itself — the tables an environment_id would point INTO — plus
// the migration ledger, which predates all of it.
const SPINE: ReadonlyArray<readonly [string, string]> = [
["organisations", "the root of the tenancy tree; nothing is above it to scope to"],
["applications", "belongs to an organisation, which is the scope"],
["environments", "IS the scope — an environment_id here would point at itself"],
[
"humans",
"a person, not a tenant's record; one human may belong to several organisations",
],
["memberships", "joins humans to organisations, above the environment level"],
[
"schema_migrations",
"the migration ledger; it predates tenancy and belongs to the database",
],
];
/** The spine, for anyone who needs to state it rather than derive it. */
export const SPINE_TABLES: readonly string[] = SPINE.map(([t]) => t);
export interface CatalogueRow extends Record<string, unknown> {
table_name: string;
has_environment_id: boolean;
fk_targets: string[] | null;
}
/** Every base table in `public`, each classified into exactly one of the three paths —
* or into none, which is the answer that fails a build. */
export async function classifyTables(db: Db): Promise<TableClassification[]> {
const rows = (
await db.execute<CatalogueRow>(sql`
WITH base AS (
SELECT table_name
FROM information_schema.tables
WHERE table_schema = 'public' AND table_type = 'BASE TABLE'
),
direct AS (
SELECT table_name
FROM information_schema.columns
WHERE table_schema = 'public' AND column_name = 'environment_id'
)
SELECT
b.table_name,
(b.table_name IN (SELECT table_name FROM direct)) AS has_environment_id,
(
-- ::text is load-bearing. information_schema columns are sql_identifier,
-- and node-pg has no parser for an array of them: the row arrives as the
-- literal string {channels,users} and iterating it yields a brace, which
-- is how this was found rather than reasoned about.
SELECT array_agg(DISTINCT ccu.table_name::text)
FROM information_schema.table_constraints tc
JOIN information_schema.constraint_column_usage ccu
ON ccu.constraint_name = tc.constraint_name
AND ccu.table_schema = tc.table_schema
WHERE tc.constraint_type = 'FOREIGN KEY'
AND tc.table_schema = 'public'
AND tc.table_name = b.table_name
AND ccu.table_name IN (SELECT table_name FROM direct)
) AS fk_targets
FROM base b
ORDER BY b.table_name
`)
).rows;
return rows.map(classifyRow);
}
/** THE CLASSIFICATION, SEPARATED FROM THE QUERY, and separated for a reason worth
* stating: the interesting arm is the one that returns `null`, and it cannot execute
* against a real database that has no unclassified table — which is exactly the state
* this check exists to keep. So the branch that fires only when somebody adds a table
* is the one branch a live run can never reach.
*
* Pure, so a unit test can drive all four arms with rows it makes up. A file with
* nothing to mock has no reason to be partially tested. */
export function classifyRow(row: CatalogueRow): TableClassification {
const via = row.fk_targets ?? [];
// ORDER MATTERS, AND ONLY IN ONE PLACE: a spine table with no environment_id and no
// foreign key classifies the same either way, but checking `direct` first means a
// future spine table that GAINS the column reports as `direct` and its list entry
// becomes visibly wrong rather than silently ignored.
if (row.has_environment_id) return { table: row.table_name, path: "direct", via };
if (via.length > 0) return { table: row.table_name, path: "hop", via };
const reason = new Map(SPINE).get(row.table_name);
if (reason !== undefined) return { table: row.table_name, path: "spine", via, reason };
return { table: row.table_name, path: null, via };
}Read from information_schema rather than from schema.ts, because a table a migration
added and Drizzle never modelled is invisible to a check that reads the model — and that
is exactly the table this exists to catch.
import { afterAll, beforeAll, describe, expect, it } from "vitest";
import { classifyTables, SPINE_TABLES, type TableClassification } from "../db/catalogue";
import { createDb, createPool } from "../db/client";
import type { Db } from "../db/client";
// THE STRUCTURAL HALF (FR-012, constitution I).
//
// The gauntlet attacks the endpoints that exist. This asks about THE LEAK THAT HAS NO
// ENDPOINT YET: a table with no path back to an environment is exposed by the first
// query that joins it, and nothing before that moment fails.
//
// WHAT THIS DOES NOT CHECK, and the distinction is easy to lose: that every table HAS a
// tenant path, not that every QUERY respects the one it has. The gauntlet makes the
// second claim, endpoint by endpoint. Neither implies the other.
describe("every table has a path to one tenant", () => {
// `ReturnType` rather than `import type pg from "pg"`: the driver's own types are
// behind the same ban as the driver, and a type-only import is still an import to
// `no-restricted-imports`.
let pool: ReturnType<typeof createPool>;
let db: Db;
let tables: TableClassification[];
beforeAll(async () => {
pool = createPool();
db = createDb(pool);
tables = await classifyTables(db);
}, 30_000);
afterAll(async () => {
await pool?.end();
});
it("classifies every base table as direct, hop or spine", () => {
const unclassified = tables.filter((t) => t.path === null).map((t) => t.table);
// The message names the tables, because the useful half of this failure is WHICH
// table appeared — a new migration's, almost always.
expect(
unclassified,
`these tables have no path to an environment: ${unclassified.join(", ")}. ` +
`Add environment_id, add a foreign key to a table that has one, or add it to ` +
`SPINE in db/catalogue.ts with a reason.`,
).toEqual([]);
});
it("classifies each table exactly once", () => {
const seen = new Set<string>();
for (const t of tables) {
expect(seen.has(t.table), `${t.table} classified twice`).toBe(false);
seen.add(t.table);
}
expect(seen.size).toBe(tables.length);
});
it("has no spine entry for a table that does not exist", () => {
// THE DIRECTION THAT CATCHES A DROPPED TABLE. A spine entry outliving its table is
// an exemption standing over nothing — harmless today, and waiting to cover a
// future table that happens to reuse the name.
const present = new Set(tables.map((t) => t.table));
const stale = SPINE_TABLES.filter((t) => !present.has(t));
expect(stale, `spine names tables that do not exist: ${stale.join(", ")}`).toEqual([]);
});
it("gives every spine table a reason", () => {
const reasonless = tables
.filter((t) => t.path === "spine" && (t.reason ?? "").trim() === "")
.map((t) => t.table);
expect(reasonless).toEqual([]);
});
it("reports the shape of the schema", () => {
const by = (p: TableClassification["path"]) => tables.filter((t) => t.path === p);
const direct = by("direct");
const hop = by("hop");
const spine = by("spine");
console.log(
`tenant paths: ${tables.length} tables — ${direct.length} direct, ` +
`${hop.length} hop, ${spine.length} spine`,
);
expect(direct.length + hop.length + spine.length).toBe(tables.length);
// A hop with no target is a hop in name only, and the query that produced it
// would have to be wrong for this to happen — which is why it is asserted.
for (const t of hop) {
expect(t.via.length, `${t.table} is a hop to nowhere`).toBeGreaterThan(0);
}
});
});tenant paths: 11 tables — 3 direct, 2 hop, 6 spineA WebSocket does not appear in router.stack, so everything above is blind to it. The
socket gets its own half — and its attack surface is derived the same way, from the
protocol's own frame union:
import { spawn, type ChildProcess } from "node:child_process";
import { randomUUID } from "node:crypto";
import { createRequire } from "node:module";
import { dirname, join } from "node:path";
import { fileURLToPath } from "node:url";
/** Two tenants and a running api, for the socket half of the gauntlet.
*
* SEEDING GOES THROUGH THE API'S OWN REPOSITORY, imported from its build output — the
* test-only seam chapter 2.8 established, for the same reason: there is no admin API for
* environments or keys, and inventing one for a test would be inventing product. The
* gateway does not depend on the api package and must not start.
*
* THE API RUNS AS A CHILD AND THE GATEWAY IN PROCESS. In process, because a test that
* cannot reach the gateway's own state cannot check what it subscribed to; as a child,
* because importing the api would make this service depend on the api's framework to
* test itself, and not knowing how the api is built is the whole of ADR-05. */
const HERE = dirname(fileURLToPath(import.meta.url));
const REPO = join(HERE, "..", "..", "..");
const require_ = createRequire(import.meta.url);
interface Seeder {
createEnvironment: (db: unknown, input: { name: string }) => Promise<{ id: string }>;
createApiKey: (
db: unknown,
input: { environmentId: string },
) => Promise<{ credential: string }>;
Repository: new (
db: unknown,
environmentId: string,
) => {
createUser: (externalId: string, name?: string) => Promise<{ id: string }>;
createChannel: (
externalId: string,
type: string,
name?: string,
) => Promise<{ id: string }>;
addMember: (channelId: string, userId: string) => Promise<boolean>;
sendMessage: (
channelId: string,
input: { text: string; userId?: string },
) => Promise<{ id: string }>;
};
}
export interface SocketTenant {
environmentId: string;
credential: string;
userExternalId: string;
userId: string;
channelId: string;
}
export interface SocketTenants {
/** The caller. Its token is the one every attack presents. */
attacker: SocketTenant;
/** The tenant whose identifiers the attacker borrows. */
victim: SocketTenant;
apiUrl: string;
stop: () => void;
}
/** THE PORT COMES FROM THE CHILD, NOT FROM A TABLE.
*
* The api is started with `PORT=0` and reports the port it bound. A hand-allocated band
* per suite is a table nothing checks: two suites eventually overlap, or a band grows to
* contain a port the lane itself runs, and the failure is a health check that succeeds
* against the wrong service. Asking the operating system removes the table. */
async function startApi(): Promise<{ url: string; stop: () => void }> {
const dist = join(REPO, "services", "api", "dist");
const child: ChildProcess = spawn("node", [join(dist, "main.js")], {
env: { ...process.env, PORT: "0" },
stdio: ["ignore", "pipe", "pipe"],
});
const port = await new Promise<number>((resolve, reject) => {
const timer = setTimeout(() => reject(new Error("api never reported a port")), 30_000);
let buffered = "";
child.stdout?.on("data", (chunk: Buffer) => {
buffered += chunk.toString();
for (const line of buffered.split("\n")) {
if (!line.trim()) continue;
try {
const parsed = JSON.parse(line) as { msg?: string; port?: number };
if (parsed.msg === "listening" && typeof parsed.port === "number") {
clearTimeout(timer);
resolve(parsed.port);
return;
}
} catch {
/* a partial line; the next chunk completes it */
}
}
});
child.on("exit", (code) => {
clearTimeout(timer);
reject(new Error(`api exited before listening (code ${String(code)})`));
});
});
return { url: `http://127.0.0.1:${port}`, stop: () => child.kill() };
}
export async function seedSocketTenants(): Promise<SocketTenants> {
const dist = join(REPO, "services", "api", "dist");
const client = require_(join(dist, "db", "client.js")) as {
createDb: (pool: unknown) => unknown;
createPool: () => unknown;
};
const seeder = require_(join(dist, "db", "repository.js")) as Seeder;
const db = client.createDb(client.createPool());
const seed = async (label: string): Promise<SocketTenant> => {
const environment = await seeder.createEnvironment(db, {
name: `socket-isolation-${label}-${randomUUID().slice(0, 8)}`,
});
const repo = new seeder.Repository(db, environment.id);
const userExternalId = `${label}-user`;
const user = await repo.createUser(userExternalId, `${label} user`);
const channel = await repo.createChannel(`${label}-channel`, "public");
await repo.addMember(channel.id, user.id);
await repo.sendMessage(channel.id, { text: `${label} says something`, userId: user.id });
const key = await seeder.createApiKey(db, { environmentId: environment.id });
return {
environmentId: environment.id,
credential: key.credential,
userExternalId,
userId: user.id,
channelId: channel.id,
};
};
const attacker = await seed("attacker");
const victim = await seed("victim");
const api = await startApi();
return { attacker, victim, apiUrl: api.url, stop: api.stop };
}
/** A token for one tenant's user, minted with that tenant's key. */
export async function mintToken(
apiUrl: string,
credential: string,
user: string,
): Promise<string> {
const res = await fetch(`${apiUrl}/auth/dev-token`, {
method: "POST",
headers: { authorization: `Bearer ${credential}`, "content-type": "application/json" },
body: JSON.stringify({ user }),
});
if (!res.ok) throw new Error(`dev-token: ${res.status}`);
return ((await res.json()) as { token: string }).token;
}The api runs as a child process on PORT=0 and reports the port it bound. A fixed port
races whichever sibling suite also binds one, and a previous run's child still holding it
makes a health check succeed against a service that has never heard of this run's data —
three unrelated-looking assertions, one fixture. Asking the operating system removes the
table that would otherwise need maintaining.
import { randomUUID } from "node:crypto";
import type { Server } from "node:http";
import type { AddressInfo } from "node:net";
import { docsUrl, frameSchema } from "@relay/protocol";
import { createLogger, serve, type Logger } from "@relay/service-kit";
import { WebSocket } from "ws";
import { afterAll, beforeAll, describe, expect, it } from "vitest";
import { createApiClient } from "./api-client.js";
import { mintToken, seedSocketTenants, type SocketTenants } from "./isolation-fixtures.js";
import { attachSessions } from "./session.js";
// THE SOCKET HALF OF THE GAUNTLET (FR-007, NFR-SEC-09, constitution I).
//
// The api's gauntlet attacks every HTTP route. None of it reaches here: A WEBSOCKET IS
// NOT IN `router.stack`, so the derived target list cannot see it, and this file is the
// only place the socket gets attacked with another tenant's identifiers.
//
// The list of things to attack is derived here too, from `frameSchema`'s own members
// rather than from a list somebody typed — same argument as the router, one protocol
// down. A frame type added to the union and forgotten here is exactly the case that
// would otherwise ship unattacked.
const silent: Logger = createLogger("gateway", () => {});
/** Every frame type the protocol declares, read off the union.
*
* ASSERTED NON-EMPTY BEFORE IT IS USED, for the reason the router derivation records: a
* shape change in zod that made this return nothing would leave the suite green while
* it attacked no frames at all. */
function declaredFrameTypes(): string[] {
const options = (frameSchema as unknown as { options?: unknown[] }).options ?? [];
const types: string[] = [];
for (const option of options) {
const shape = (option as { shape?: { type?: { value?: unknown } } }).shape;
const value = shape?.type?.value;
if (typeof value === "string") types.push(value);
}
return types;
}
async function closeCode(socket: WebSocket): Promise<number> {
return new Promise((resolve, reject) => {
socket.on("close", (code) => resolve(code));
socket.on("error", () => undefined);
setTimeout(() => reject(new Error("no close within 5s")), 5_000);
});
}
async function firstFrame(socket: WebSocket, type: string): Promise<Record<string, unknown>> {
return new Promise((resolve, reject) => {
socket.on("message", (raw) => {
const frame = JSON.parse(raw.toString()) as Record<string, unknown>;
if (frame.type === type) resolve(frame);
});
socket.on("close", (code) => reject(new Error(`closed ${code}`)));
setTimeout(() => reject(new Error(`no ${type} within 5s`)), 5_000);
});
}
describe("the socket refuses another tenant's identifiers", () => {
let t: SocketTenants;
let server: Server;
let url: string;
let attackerToken: string;
beforeAll(async () => {
t = await seedSocketTenants();
server = serve({
service: "gateway",
health: () => ({}),
logger: silent,
notFoundDocsUrl: docsUrl("not_found"),
});
attachSessions({ server, api: createApiClient(t.apiUrl), logger: silent });
await new Promise<void>((resolve) => server.listen(0, resolve));
url = `ws://127.0.0.1:${(server.address() as AddressInfo).port}`;
attackerToken = await mintToken(t.apiUrl, t.attacker.credential, t.attacker.userExternalId);
}, 90_000);
afterAll(async () => {
await new Promise<void>((resolve) => server?.close(() => resolve()));
t?.stop();
});
it("derives the frame types from the protocol, and finds some", () => {
const types = declaredFrameTypes();
// An empty derivation is a broken derivation, not a small protocol.
expect(types.length).toBeGreaterThan(1);
expect(types).toContain("message.send");
});
it("a connection ack names nothing belonging to the other tenant", async () => {
const socket = new WebSocket(`${url}/v1/ws?token=${attackerToken}`);
const ack = await firstFrame(socket, "connection.ack");
const serialised = JSON.stringify(ack);
expect(serialised).not.toContain(t.victim.channelId);
expect(serialised).not.toContain(t.victim.environmentId);
expect(serialised).not.toContain(t.victim.userId);
socket.close();
}, 20_000);
it("message.send to the other tenant's channel is refused", async () => {
const socket = new WebSocket(`${url}/v1/ws?token=${attackerToken}`);
await firstFrame(socket, "connection.ack");
socket.send(
JSON.stringify({
type: "message.send",
payload: {
idem_key: randomUUID(),
channel: t.victim.channelId,
text: "from the attacker",
},
}),
);
const error = await firstFrame(socket, "error");
const payload = error.payload as { code?: string; message?: string };
// The refusal must not name what it refused. An error that echoes the channel id
// back tells the attacker the channel exists, which is the leak the HTTP gauntlet
// proves is absent on every route — the socket does not get an exemption.
expect(JSON.stringify(payload)).not.toContain(t.victim.channelId);
expect(payload.code).toBeTruthy();
socket.close();
}, 20_000);
it("every declared frame type that is not message.send is refused inbound", async () => {
// SCHEMA VALIDATION RUNS BEFORE THE TYPE CHECK, and that shapes what this can
// claim. A frame whose payload does not match its own schema is answered
// `invalid_frame` and never reaches the rule that says clients may not utter a
// server frame — so a loop sending `{}` for every type would pass while testing
// the parser, not the rule. The first draft of this test did exactly that.
//
// So: every declared type must be refused SOMEHOW, and one well-formed server
// frame must be refused BY THE RULE.
const inboundOnly = declaredFrameTypes().filter((type) => type !== "message.send");
expect(inboundOnly.length).toBeGreaterThan(0);
for (const type of inboundOnly) {
const socket = new WebSocket(`${url}/v1/ws?token=${attackerToken}`);
await firstFrame(socket, "connection.ack");
socket.send(JSON.stringify({ type, payload: {} }));
const error = await firstFrame(socket, "error");
expect((error.payload as { code?: string }).code, `${type} was not refused`).toBeTruthy();
socket.close();
}
// `message.ack` carries `{ seq }`, which is the easiest valid server frame to
// build — so it is the one that proves the rule rather than the parser.
const socket = new WebSocket(`${url}/v1/ws?token=${attackerToken}`);
await firstFrame(socket, "connection.ack");
socket.send(JSON.stringify({ type: "message.ack", payload: { seq: 1 } }));
const error = await firstFrame(socket, "error");
expect((error.payload as { code?: string }).code).toBe("unknown_frame_type");
// A protocol violation closes the connection (EIR-WS-06's 4002).
await expect(closeCode(socket)).resolves.toBe(4002);
}, 60_000);
it("a token minted by one tenant cannot open a session for the other", async () => {
// The attacker asks its OWN api for a token naming the victim's user. Either the
// mint refuses, or the session it opens must resolve nothing of the victim's.
const borrowed = await mintToken(
t.apiUrl,
t.attacker.credential,
t.victim.userExternalId,
).catch(() => null); // refused at the mint is the stronger answer
if (borrowed === null) return;
const socket = new WebSocket(`${url}/v1/ws?token=${borrowed}`);
try {
const ack = await firstFrame(socket, "connection.ack");
expect(JSON.stringify(ack)).not.toContain(t.victim.channelId);
} catch {
// A closed socket is a refusal, which is also correct.
await expect(closeCode(socket)).resolves.toBeGreaterThan(0);
}
socket.close();
}, 20_000);
});Both of the socket tests above were green before they were correct, and the way they were wrong is worth more than the way they are right.
The cross-tenant send used a payload of { channel_id, text }. The real frame carries
{ idem_key, channel, text }, so the socket answered invalid_frame — "expected string, received undefined" and refused it as malformed, before any tenant check ran. The
test asserted that the reply was a refusal and did not name the victim's channel. Both
were true. Neither had anything to do with isolation.
The frame-union loop had the same shape. Schema validation runs before the type check, so
sending {} for every server frame type gets invalid_frame and never reaches the rule
that says clients may not utter a server frame. The loop was testing the parser.
The fix for the second one is not to delete the loop — every declared type should still
be refused somehow — but to add one well-formed server frame, message.ack with its
{ seq }, that reaches the rule and proves it.
Three changes this chapter needed and did not set out to make.
The integration suites could not run together at all. Three of nine api suites failed
with duplicate key value violates unique constraint "pg_type_typname_nsp_index" —
concurrent migrations racing to create the same type. Three of three concurrent runs
failed; two of two serialised runs passed. Reproducible, not flaky:
@@ -9,13 +9,13 @@
"scripts": {
"dev": "turbo run dev",
"lint": "turbo run //#lint:root",
"lint:root": "eslint .",
"typecheck": "turbo run typecheck",
"test": "turbo run test",
- "test:integration": "turbo run test:integration",
+ "test:integration": "turbo run test:integration --concurrency=1",
"build": "turbo run build"
},
"devDependencies": {
"@eslint/js": "^10.0.1",
"@types/node": "^26.1.2",
"eslint": "^10.8.0",The api logged the port it asked for, which is 0:
@@ -7,12 +7,25 @@ import { AppModule } from "./app.module";
// Nest's own banner logger stays off: this workspace already decided what a
// log line looks like (one JSON object, NFR-OBS-01), and the framework does
// not get a second opinion.
async function bootstrap(): Promise<void> {
const app = await NestFactory.create(AppModule, { logger: false });
- const port = Number(process.env.PORT ?? 4000);
- await app.listen(port);
+ const requested = Number(process.env.PORT ?? 4000);
+ await app.listen(requested);
+ // THE PORT IT GOT, NOT THE PORT IT ASKED FOR.
+ //
+ // `PORT=0` asks the operating system for any free port, which is what a test
+ // spawning this service should do — a fixed port races whichever sibling suite also
+ // binds one, and a previous run's child still holding it makes a health check succeed
+ // against a service that has never heard of this run's data. Three unrelated-looking
+ // assertions, one fixture.
+ //
+ // But a parent can only use the number if this process reports it, and logging
+ // `requested` prints 0. So the bound address is read back and logged.
+ const address = app.getHttpServer().address() as { port?: number } | string | null;
+ const port =
+ typeof address === "object" && address !== null ? (address.port ?? requested) : requested;
createLogger("api").log("info", "listening", { port });
}
void bootstrap();And seeding two tenants broke a neighbour. signup.itest.ts counted organisations
across the whole table before and after a failed provision and asserted the two matched —
a global claim about a local operation. The failure read expected 452 to be 451, which
says nothing about a neighbour at all. The test directly above it already scopes its
questions by id:
@@ -149,37 +149,35 @@ describe("signup", () => {
expect(Number(counts.apps)).toBe(1);
expect(Number(counts.envs)).toBe(1);
expect(Number(counts.owners)).toBe(1);
});
it("writes the full set or nothing when provisioning fails (invariant 1)", async () => {
- const before = await db.execute(
- `SELECT count(*)::int AS n FROM organisations`,
- );
- // Force a failure inside the transaction, after the organisation insert:
- // an organisation name that is fine and a provider value the CHECK
- // constraint refuses.
+ // SCOPED TO THE ROW THIS TEST CREATES, not to the table.
+ //
+ // This read `SELECT count(*) FROM organisations` before and after and asserted the
+ // two matched. That is a GLOBAL claim about a LOCAL operation: any other suite that
+ // provisions a tenant between the two reads breaks it, and the failure —
+ // `expected 452 to be 451` — says nothing about a neighbour. The isolation
+ // fixtures seed two tenants and did exactly that.
+ //
+ // What invariant 1 actually claims is that the failed transaction left NOTHING
+ // behind. That is a question about one organisation, and the test above already
+ // asks its questions that way.
await expect(
provisionOrganisation(db, {
provider: "not-a-provider",
providerAccountId: "x",
organisationName: "doomed org",
}),
).rejects.toThrow();
- const after = await db.execute(
- `SELECT count(*)::int AS n FROM organisations`,
- );
- // Nothing survived — no half-built tenant, which is the whole point of the
- // single transaction.
- expect((after.rows[0] as { n: number }).n).toBe(
- (before.rows[0] as { n: number }).n,
- );
- const orphan = await db.execute(
+ const doomed = await db.execute(
`SELECT count(*)::int AS n FROM organisations WHERE name = 'doomed org'`,
);
- expect((orphan.rows[0] as { n: number }).n).toBe(0);
+ // No half-built tenant, which is the whole point of the single transaction.
+ expect((doomed.rows[0] as { n: number }).n).toBe(0);
});
it("recognises a returning owner instead of creating a second organisation (invariant 2)", async () => {
const accountId = `returning-${Date.now()}`;
const { first, second } = await withFreshIdentity(accountId, async () => ({
first: await signUp("code-a"),A cross-tenant send over the socket is answered internal_error: "send failed".
Isolation holds — it is indistinguishable from any other failure — but the code is wrong,
and a client cannot act on it. That is a refusal wearing an error's clothes, and it is
recorded rather than fixed here.
The suite has no unit tests of its own. classifyRow returns null for an unclassified
table and comparePair reports a difference, and neither arm can execute against a
healthy lane — the states they describe are exactly the ones this chapter exists to
prevent. An instrument that has never fired is an untested instrument, and that is a
chapter of its own.