Building Relay

Phần 3 · Chương 3.4

Cửa ải cô lập tenant

Bạn sẽ tạo ra: Một bộ kiểm thử cross-tenant tự suy ra danh sách mục tiêu từ router đang chạy và đỏ khi gặp route nó chưa có quyết định nào, ba dạng tấn công so sánh một CẶP phản hồi thay vì một mã trạng thái, một kiểm tra cấu trúc rằng mọi bảng đều có đường về tenant, và tầng socket bị tấn công từ chính frame union của protocol · khoảng 55 phút, bao gồm bài tập

Tài liệu gốc: SRS — Đặc tả yêu cầu phần mềm · SAD — Tài liệu kiến trúc phần mềm (tiếng Anh)

Constitution I nói data của tenant này không bao giờ được tới tenant khác. Mọi chương đến đây đều đồng ý, một vài chương đã test điều đó — messages.itest.ts kiểm tra foreign channel trả not found, và test ấy đúng.

Vấn đề của chín assertion rải rác không phải assertion nào sai. Vấn đề là không cái nào cho biết thứ gì đang thiếu. Route được thêm tuần sau mà không có isolation test trông y hệt route đã có test, vì thứ cần tìm chính là một sự vắng mặt.

Vì vậy chương này dựng suite không thể mắc lỗi đó: tự suy ra thứ cần tấn công từ application đang chạy và fail khi gặp thứ chưa có decision.

Danh sách phải tự suy ra chính nó

Fault cần ngăn là route tồn tại nhưng chưa bị tấn công. Chỉ router biết thứ gì tồn tại, vì vậy ta hỏi chính router.

services/api/src/isolation/targets.ts
/** The gauntlet's target list (NFR-SEC-09).
 *
 * A LIST OF CLASSIFICATIONS, NOT A LIST OF TARGETS. The targets themselves are derived
 * from the running application — `app.getHttpAdapter().getInstance().router.stack` —
 * because the fault this suite exists to prevent is a route that exists and is
 * unattacked, and only the router knows what exists. What lives here is the decision
 * about each one, which no derivation can make.
 *
 * NOTHING MAY BE EXEMPT BY OMISSION. A derived target matching no entry fails the
 * suite, and an entry matching no derived target fails it too — the second direction is
 * the one that catches a stale exemption after a rename. That pair of assertions is the
 * whole mechanism. */
 
/** What kind of attack a route takes.
 *
 * `credential` is the shape a foreign-identifier attack cannot express. `POST
 * /auth/dev-token` accepts no tenant-owned identifier, so there is nothing to put in
 * one — and it is tenant-scoped all the same, because the key it accepts resolves to
 * exactly one environment. Filing it as `exempt` is how a route stops being attacked
 * while looking accounted for. */
export type Shape = "read" | "write" | "credential" | "exempt";
 
/** Which credential class the route accepts, and therefore which attack applies.
 *
 * A `write` shape alone cannot tell these apart. A route taking an end-user token is
 * already scoped to one environment, so the attack is a FOREIGN CREDENTIAL. A route
 * taking an application key is scoped too, but by a different resolution — and a route
 * that accepts either is attacked as both, which is why `either` is a class rather
 * than a shrug. */
export type CredentialClass = "application" | "user" | "either" | "none";
 
interface Classified {
  method: string;
  path: string;
  accepts: CredentialClass;
}
 
/** `exempt` requires a reason. The type is what makes that true — an exemption with no
 * argument beside it is indistinguishable from an oversight six months later. */
export type Classification =
  | (Classified & { shape: Exclude<Shape, "exempt">; because?: string })
  | (Classified & { shape: "exempt"; because: string });
 
/** Every route the api serves, as the derivation reports it: 9 today.
 *
 * The shapes sum to 9 — 3 exempt, 2 credential, 1 read, 3 write — and that sum is
 * asserted against the derivation rather than written down twice. A count nobody
 * recomputes is a count that stops being true quietly.
 *
 * THIS LIST GROWS WITH THE PLATFORM. Every chapter after this one that adds a route
 * adds a row here, and the suite goes red until it does. That is the point of building
 * the gauntlet now rather than at the end: a target list written after the fact is a
 * list somebody reconstructs from memory, and the routes it forgets are exactly the
 * ones nobody was thinking about. */
export const CLASSIFICATIONS: readonly Classification[] = [
  // ── exempt: no tenant-owned identifier and no tenant-scoped credential ──────────
  {
    method: "GET",
    path: "/healthz",
    accepts: "none",
    shape: "exempt",
    because:
      "liveness only. No credential, no identifier, and the body is a service name and an uptime.",
  },
  {
    method: "GET",
    path: "/auth/:provider/start",
    accepts: "none",
    shape: "exempt",
    because:
      "pre-tenant. `:provider` names an identity provider, not anything a tenant owns, and the caller has no credential yet — this is the route that gets them one.",
  },
  {
    method: "GET",
    path: "/auth/:provider/callback",
    accepts: "none",
    shape: "exempt",
    because:
      "pre-tenant, and the route that CREATES the tenant. There is no second tenant to reach across from, because at this point the caller belongs to none.",
  },
 
  // ── credential: tenant-scoped, but with nothing to put a foreign id into ────────
  {
    method: "POST",
    path: "/auth/dev-token",
    accepts: "application",
    shape: "credential",
    because:
      "the body names no tenant-owned resource, so a foreign-identifier attack has nothing to express. It is tenant-scoped all the same: the key it accepts resolves to exactly one environment, and the token it mints must not outlive that scope.",
  },
 
  // ── the public message surface: attacked as both classes, because it takes both ─
  {
    method: "POST",
    path: "/v1/channels/:channelId/messages",
    accepts: "either",
    shape: "write",
  },
  {
    method: "GET",
    path: "/v1/channels/:channelId/messages",
    accepts: "either",
    shape: "read",
  },
 
  // ── the internal surface: an end-user token, so a FOREIGN CREDENTIAL is the attack
  { method: "POST", path: "/internal/messages", accepts: "user", shape: "write" },
  { method: "POST", path: "/internal/backfill", accepts: "user", shape: "write" },
  {
    // NOT A `write`, AND THE DIFFERENCE IS THE WHOLE POINT OF HAVING SHAPES. This route
    // takes no body and no path parameter: there is no identifier to forge, so a
    // foreign-identifier attack has nothing to express. Its only tenant-scoped input is
    // the token, which is what `credential` attacks. Classifying it `write` would have
    // produced an attack that sends a valid request and proves nothing.
    method: "POST",
    path: "/internal/session",
    accepts: "user",
    shape: "credential",
  },
];
 
export function targetKey(t: { method: string; path: string }): string {
  return `${t.method.toUpperCase()} ${t.path}`;
}
 
/** Counts, for the suite to print. Derived from the list rather than typed beside it,
 * because a hand-maintained tally is the thing that goes stale first. */
export function shapeCounts(list: readonly Classification[]): Record<Shape, number> {
  // NO `list` SHAPE YET, and that is deliberate rather than an omission. Nothing this
  // api serves returns a collection, so a list attack would be a function with no
  // target — and a shape with no member is a vocabulary entry that drifts. The chapter
  // that adds the first list route adds the shape and the attack together.
  const counts: Record<Shape, number> = { read: 0, write: 0, credential: 0, exempt: 0 };
  for (const c of list) counts[c.shape]++;
  return counts;
}
 
/** One routable endpoint, as the running application reports it. */
export interface DerivedTarget {
  method: string;
  path: string;
}
 
/** The shape of the express router this reaches into. Declared rather than imported:
 * express 5 ships no types, and adding `@types/express` for one property read would
 * move the api's dependency list for a test's benefit. */
interface RouterLike {
  stack?: Array<{
    route?: { path?: string; methods?: Record<string, boolean> };
    name?: string;
  }>;
}
interface AdapterInstance {
  router?: RouterLike;
  _router?: RouterLike;
}
 
/** Where the router lives, and the fact that this is not public API.
 *
 * Express 5 exposes `router`; express 4 called it `_router`. Both are read, and WHICH
 * ONE ANSWERED is returned so a test can assert the derivation found something rather
 * than silently finding nothing. That is the failure mode worth designing against: a
 * renamed property would leave this suite green while it attacked zero routes, which is
 * worse than the hand-written list it replaces. */
export function deriveTargets(instance: unknown): {
  targets: DerivedTarget[];
  middlewareLayers: number;
  property: "router" | "_router" | "none";
} {
  const adapter = instance as AdapterInstance;
  const router = adapter.router ?? adapter._router;
  const property = adapter.router ? "router" : adapter._router ? "_router" : "none";
  const targets: DerivedTarget[] = [];
  let middlewareLayers = 0;
  for (const layer of router?.stack ?? []) {
    if (!layer.route) {
      middlewareLayers++;
      continue;
    }
    const path = layer.route.path ?? "";
    for (const [verb, on] of Object.entries(layer.route.methods ?? {})) {
      if (on) targets.push({ method: verb.toUpperCase(), path });
    }
  }
  return { targets, middlewareLayers, property };
}
 
/** A route that has existed since chapter 2.2 and will exist for as long as this
 * product does. If the derivation cannot find THIS, it has not found the mounted
 * router, whatever else it returned. */
export const CANARY_TARGET = "POST /v1/channels/:channelId/messages";
services/api/src/isolation/targets.itest.ts
import "reflect-metadata";
 
import type { INestApplication } from "@nestjs/common";
import { Test } from "@nestjs/testing";
import { afterAll, beforeAll, describe, expect, it } from "vitest";
 
import { AppModule } from "../app.module";
import {
  CANARY_TARGET,
  CLASSIFICATIONS,
  deriveTargets,
  shapeCounts,
  targetKey,
  type DerivedTarget,
} from "./targets";
 
// THE DERIVATION'S OWN TESTS, and they matter more than they look.
//
// The gauntlet attacks a list it derives from the running application rather than one
// somebody typed, because the fault NFR-SEC-09 exists to prevent is a route that exists
// and is unattacked. But a derivation has a failure mode a typed list does not: IT CAN
// RETURN NOTHING AND PASS. Express 5 renamed `_router` to `router`; an upgrade that
// renamed it again would leave this suite green while it attacked zero endpoints, which
// is worse than the hand-written list it replaces, because a hand-written list at least
// looks wrong when you read it.
//
// So the derivation is checked before it is used: it found a router, it found routes, it
// found the route we know is there, and every target it found is classified exactly once.
 
describe("the gauntlet's target list derives from the running application", () => {
  let app: INestApplication;
  let derived: DerivedTarget[];
  let middlewareLayers: number;
  let property: string;
 
  beforeAll(async () => {
    const moduleRef = await Test.createTestingModule({ imports: [AppModule] }).compile();
    app = moduleRef.createNestApplication();
    await app.init();
    const result = deriveTargets(app.getHttpAdapter().getInstance());
    derived = result.targets;
    middlewareLayers = result.middlewareLayers;
    property = result.property;
  }, 60_000);
 
  afterAll(async () => {
    await app?.close();
  });
 
  it("found a router at all, and says which property answered", () => {
    expect(property).not.toBe("none");
    expect(["router", "_router"]).toContain(property);
  });
 
  it("found routes — an empty list is a broken derivation, not a clean surface", () => {
    expect(derived.length).toBeGreaterThan(0);
    // Middleware layers exist in any mounted express app. Zero of them alongside zero
    // routes is the signature of reading the wrong object rather than of a small api.
    expect(middlewareLayers).toBeGreaterThan(0);
  });
 
  it("found the route that has existed since chapter 2.2", () => {
    expect(derived.map(targetKey)).toContain(CANARY_TARGET);
  });
 
  it("classifies every derived target exactly once", () => {
    const entries = new Map<string, number>();
    for (const c of CLASSIFICATIONS) {
      entries.set(targetKey(c), (entries.get(targetKey(c)) ?? 0) + 1);
    }
    const unclassified = derived.map(targetKey).filter((k) => !entries.has(k));
    const duplicated = [...entries].filter(([, n]) => n > 1).map(([k]) => k);
    // Named in the failure, because "9 !== 10" sends a reader to count rows.
    expect({ unclassified, duplicated }).toEqual({ unclassified: [], duplicated: [] });
  });
 
  it("has no classification entry that matches no derived target", () => {
    const found = new Set(derived.map(targetKey));
    const stale = CLASSIFICATIONS.map(targetKey).filter((k) => !found.has(k));
    // THIS IS THE DIRECTION THAT CATCHES A RENAME. A route renamed with its exemption
    // left behind is a route nobody attacks and nobody misses.
    expect(stale).toEqual([]);
  });
 
  it("gives every exempt entry a reason", () => {
    const reasonless = CLASSIFICATIONS.filter(
      (c) => c.shape === "exempt" && (!c.because || c.because.trim() === ""),
    ).map(targetKey);
    expect(reasonless).toEqual([]);
  });
 
  it("accounts for every derived target as attacked or exempt", () => {
    const counts = shapeCounts(CLASSIFICATIONS);
    const attacked = counts.read + counts.write + counts.credential;
    // A number nobody can see is a number nobody checks. Visible under
    // `--reporter=verbose`; the assertion below is what gates the build either way.
    console.log(
      `gauntlet targets: ${derived.length} derived, ${attacked} attacked, ${counts.exempt} exempt ` +
        `(read ${counts.read}, write ${counts.write}, credential ${counts.credential})`,
    );
    expect(attacked + counts.exempt).toBe(derived.length);
  });
});
the derivation, reporting itself
gauntlet targets: 9 derived, 6 attacked, 3 exempt (read 1, write 3, credential 2)

File đó không chứa danh sách target mà chứa danh sách decision — mỗi route một decision nói attack nào áp dụng; nếu exempt thì vì sao. Target được lấy từ router.stack lúc runtime.

Vì vậy derivation báo property nào đã trả lời; ba test đầu kiểm tra derivation chứ chưa kiểm tra isolation.

Chạy nó sẽ cho biết đã tìm thấy gì. Hiện có chín route. Mọi chương sau thêm route cũng thêm một row vào danh sách; suite đỏ cho tới khi row xuất hiện. Đó là lý do dựng gauntlet bây giờ thay vì cuối Part: target list viết sau là danh sách ai đó tái dựng từ trí nhớ, và route bị quên chính là route không ai nghĩ đến.

Hai tenant và không có gì bị xoá

Attack cần victim. Một environment chỉ chứng minh UUID bịa ra trả not found — claim đó yếu hơn rất nhiều so với constitution.

services/api/src/isolation/fixtures.ts
import { createApiKey, createEnvironment, Repository } from "../db/repository";
 
import type { Db } from "../db/client";
 
/** Two tenants, so every attack has a victim and an attacker.
 *
 * The gauntlet's unit of assertion is a PAIR of requests — another tenant's identifier
 * and an identifier that exists nowhere — and it needs a second tenant to borrow
 * identifiers from. One environment can only prove that a made-up uuid is not found,
 * which is a much weaker claim than the one constitution I makes.
 *
 * EVERY ROW IS SCOPED TO AN ENVIRONMENT THIS FUNCTION MINTED, AND NOTHING HERE DELETES
 * ANYTHING. A fixture that tidied up with a `DELETE FROM channels` would be a global
 * operation asserting a local fact, and the lane shares one database — that is the
 * trade chapter 2.1 made when it put `environment_id` in every table, and it pays for
 * itself here. The environments are disposable; leaving them costs rows and no
 * correctness.
 *
 * ONE OF EACH KIND OF ROW THE ROUTES TOUCH, because the write attacks read storage
 * before and after: a channel and a message for the message routes, a user for the
 * session route. */
export interface Tenant {
  environmentId: string;
  /** An `rk_dev_…` credential for this environment, minted the way signup does. */
  credential: string;
  userId: string;
  userExternalId: string;
  channelId: string;
  /** The customer-supplied identifier, so an attack can present the other tenant's own
   * external id rather than only its uuid. */
  channelExternalId: string;
  messageId: string;
  repo: Repository;
}
 
export interface TwoTenants {
  /** The caller. Its credential is the one every attack presents. */
  attacker: Tenant;
  /** The tenant whose identifiers the attacker borrows. Nothing it owns may move. */
  victim: Tenant;
}
 
async function seedTenant(db: Db, label: string): Promise<Tenant> {
  const environment = await createEnvironment(db, { name: `isolation-${label}` });
  const key = await createApiKey(db, { environmentId: environment.id });
  const repo = new Repository(db, environment.id);
 
  const userExternalId = `${label}-user`;
  const user = await repo.createUser(userExternalId, `${label} user`);
  const channelExternalId = `${label}-channel`;
  const channel = await repo.createChannel(channelExternalId, "public", label);
  await repo.addMember(channel.id, user.id);
  const message = await repo.sendMessage(channel.id, {
    text: `${label} says something`,
    userId: user.id,
  });
 
  return {
    environmentId: environment.id,
    credential: key.credential,
    userId: user.id,
    userExternalId,
    channelId: channel.id,
    channelExternalId,
    messageId: message.id,
    repo,
  };
}
 
/** Seeded sequentially rather than with `Promise.all`, so a failure names which tenant
 * failed to seed instead of rejecting whichever lost the race. */
export async function seedTwoTenants(db: Db): Promise<TwoTenants> {
  const attacker = await seedTenant(db, `attacker-${Date.now().toString(36)}`);
  const victim = await seedTenant(db, `victim-${Date.now().toString(36)}`);
  return { attacker, victim };
}

Fixture không xoá gì, có chủ ý. Fixture cleanup bằng DELETE FROM channels sẽ thay đổi global state để assert local fact — lane dùng chung database và row của mọi suite khác đều ở đó. Để lại environment tốn row nhưng không tốn correctness; đó là trade-off 2.1 chọn khi đặt environment_id trong mọi table.

Một cặp, không phải status code

Đây là design decision toàn suite dựa vào.

Assertion hiển nhiên là expect(res.status).toBe(404). Nó sai ba cách cùng lúc: đóng băng status hiện tại trong security test nên thay đổi status có cân nhắc cũng làm suite fail dù security không đổi; không nói gì về body; và vẫn pass endpoint làm lộ data qua prose.

services/api/src/isolation/attack.ts
/** The three attacks, one per shape (NFR-SEC-09).
 *
 * THE UNIT OF ASSERTION IS A PAIR, NOT A REQUEST. Constitution I forbids revealing
 * that another tenant's data exists, so the correct answer to a foreign identifier is
 * whatever the platform says about an identifier that exists NOWHERE — and a single
 * response cannot show that. Every attack here issues both and compares them.
 *
 * A suite asserting `404` instead would be wrong in three ways at once. It would
 * freeze today's status choices into a test, so a considered change to a status breaks
 * a security suite for no security reason. It would say nothing about the body. And it
 * would PASS AN ENDPOINT THAT LEAKS THROUGH ITS PROSE — an error message that echoes
 * the identifier back makes the foreign answer differ from the absent one while both
 * are 404, which is exactly the leak constitution I is about.
 *
 * So status and whole body are compared. There is nothing to exclude from the
 * comparison yet: the error envelope is `code`, `message` and `docs_url`, all three of
 * which must match. When a per-request field joins it, the chapter that adds it owns
 * the decision to drop it here — and it will have to argue that the field reveals
 * nothing about the resource. */
 
export interface AttackRequest {
  method: string;
  /** Path with identifiers already substituted — `/v1/channels/<uuid>/messages`. */
  path: string;
  body?: unknown;
}
 
export interface Answer {
  status: number;
  body: unknown;
}
 
/** What an attack found. `differences` is empty when the pair is indistinguishable;
 * when it is not, it says WHAT differed, because "expected true to be false" sends a
 * reader back to the source and a named difference does not. */
export interface Verdict {
  differences: string[];
  foreign: Answer;
  absent: Answer;
}
 
async function send(
  baseUrl: string,
  credential: string,
  req: AttackRequest,
): Promise<Answer> {
  const res = await fetch(`${baseUrl}${req.path}`, {
    method: req.method,
    headers: {
      authorization: `Bearer ${credential}`,
      ...(req.body === undefined ? {} : { "content-type": "application/json" }),
    },
    ...(req.body === undefined ? {} : { body: JSON.stringify(req.body) }),
  });
  const text = await res.text();
  let body: unknown = text;
  try {
    body = text === "" ? null : JSON.parse(text);
  } catch {
    /* a non-JSON body is itself the answer, and comparing it verbatim is correct */
  }
  return { status: res.status, body };
}
 
/** Exported so a unit test can drive it. THE ARM THAT REPORTS A DIFFERENCE NEVER
 * EXECUTES IN A HEALTHY LANE — every attack in the gauntlet compares equal — so the one
 * branch that matters here is the one a passing suite cannot reach. Same shape as
 * `classifyRow`: an instrument that has never fired is an untested instrument. */
export function comparePair(foreign: Answer, absent: Answer): string[] {
  const differences: string[] = [];
  if (foreign.status !== absent.status) {
    differences.push(`status ${foreign.status} (foreign) vs ${absent.status} (absent)`);
  }
  const f = JSON.stringify(foreign.body);
  const a = JSON.stringify(absent.body);
  if (f !== a) differences.push(`body ${f} (foreign) vs ${a} (absent)`);
  return differences;
}
 
/** A read of another tenant's resource must answer as a read of nothing. */
export async function readAttack(
  baseUrl: string,
  credential: string,
  foreignReq: AttackRequest,
  absentReq: AttackRequest,
): Promise<Verdict> {
  const foreign = await send(baseUrl, credential, foreignReq);
  const absent = await send(baseUrl, credential, absentReq);
  return { differences: comparePair(foreign, absent), foreign, absent };
}
 
/** A write against another tenant's identifier must change nothing, and the pair must
 * still be indistinguishable.
 *
 * THE STATE READ IS THE POINT. A 404 that COMPLETED the write is the case no status
 * code reveals, and it is the one a reader should worry about. `readVictimState` is
 * supplied by the caller and goes through `Repository` methods rather than raw SQL: the
 * lint ban forbids the query engine outside `services/api/src/db`, and this suite
 * should not need an exemption to do its job. */
export async function writeAttack(
  baseUrl: string,
  credential: string,
  foreignReq: AttackRequest,
  absentReq: AttackRequest,
  readVictimState: () => Promise<unknown>,
): Promise<Verdict & { stateChanged: boolean; before: unknown; after: unknown }> {
  const before = await readVictimState();
  const foreign = await send(baseUrl, credential, foreignReq);
  const absent = await send(baseUrl, credential, absentReq);
  const after = await readVictimState();
  return {
    differences: comparePair(foreign, absent),
    foreign,
    absent,
    stateChanged: JSON.stringify(before) !== JSON.stringify(after),
    before,
    after,
  };
}
 
/** The shape a foreign-identifier attack cannot express.
 *
 * `POST /auth/dev-token` accepts no tenant-owned identifier, so there is nothing to
 * forge — and it is tenant-scoped all the same, because the key it accepts resolves to
 * exactly one environment. The attack is therefore ON THE CREDENTIAL: mint a token with
 * environment A's key and present it where only B's users belong. Filing this route as
 * exempt is how a route stops being attacked while looking accounted for. */
export interface CredentialVerdict {
  minted: boolean;
  /** The status the borrowed token got on the other tenant's resource. */
  crossStatus: number;
  crossBody: unknown;
}
 
export async function credentialAttack(
  baseUrl: string,
  attackerCredential: string,
  user: string,
  victimReq: AttackRequest,
): Promise<CredentialVerdict> {
  const mint = await fetch(`${baseUrl}/auth/dev-token`, {
    method: "POST",
    headers: {
      authorization: `Bearer ${attackerCredential}`,
      "content-type": "application/json",
    },
    body: JSON.stringify({ user }),
  });
  if (!mint.ok) {
    return { minted: false, crossStatus: mint.status, crossBody: await mint.json() };
  }
  const { token } = (await mint.json()) as { token: string };
  const answer = await send(baseUrl, token, victimReq);
  return { minted: true, crossStatus: answer.status, crossBody: answer.body };
}
services/api/src/isolation/gauntlet.itest.ts
import "reflect-metadata";
 
import type { INestApplication } from "@nestjs/common";
import { Test } from "@nestjs/testing";
import { afterAll, beforeAll, describe, expect, it } from "vitest";
 
import { AppModule } from "../app.module";
import { createDb, createPool } from "../db/client";
import { credentialAttack, readAttack, writeAttack } from "./attack";
import { seedTwoTenants, type TwoTenants } from "./fixtures";
import { CLASSIFICATIONS, targetKey } from "./targets";
 
import type { Db } from "../db/client";
 
// THE GAUNTLET (NFR-SEC-09, constitution I).
//
// Two tenants over real HTTP. The attacker holds a valid credential for its own
// environment and presents the victim's identifiers with it. Every attack asserts a
// PAIR — the foreign identifier and one that exists nowhere must be indistinguishable —
// because "it returned 404" is a claim about a status code and constitution I is a
// claim about what a caller can learn.
//
// AN ATTACK PER NON-EXEMPT ROUTE, AND THE SUITE PROVES IT RAN THEM. A classification
// saying `write` with no attack written for it is the same hole as a route with no
// classification, one level up. The last test in this file compares what ran against
// what the list says should have run.
 
const ABSENT_UUID = "00000000-0000-4000-8000-000000000000";
 
describe("the isolation gauntlet", () => {
  let app: INestApplication;
  let url: string;
  let db: Db;
  let pool: ReturnType<typeof createPool>;
  let t: TwoTenants;
  /** A token minted with the ATTACKER's key, for the routes that take one. */
  let attackerToken: string;
  const attacked = new Set<string>();
 
  beforeAll(async () => {
    pool = createPool();
    db = createDb(pool);
    t = await seedTwoTenants(db);
    app = (
      await Test.createTestingModule({ imports: [AppModule] }).compile()
    ).createNestApplication({ logger: false });
    await app.listen(0);
    url = await app.getUrl();
 
    const mint = await fetch(`${url}/auth/dev-token`, {
      method: "POST",
      headers: {
        authorization: `Bearer ${t.attacker.credential}`,
        "content-type": "application/json",
      },
      body: JSON.stringify({ user: t.attacker.userExternalId }),
    });
    expect(mint.ok, "the attacker must be able to mint a token for its OWN user").toBe(
      true,
    );
    attackerToken = ((await mint.json()) as { token: string }).token;
  }, 60_000);
 
  afterAll(async () => {
    await app?.close();
    await pool?.end();
  });
 
  // ── read ────────────────────────────────────────────────────────────────────────
  it("GET /v1/channels/:channelId/messages — a foreign channel reads as an absent one", async () => {
    attacked.add("GET /v1/channels/:channelId/messages");
    const verdict = await readAttack(
      url,
      t.attacker.credential,
      { method: "GET", path: `/v1/channels/${t.victim.channelId}/messages` },
      { method: "GET", path: `/v1/channels/${ABSENT_UUID}/messages` },
    );
    expect(verdict.differences, verdict.differences.join("; ")).toEqual([]);
  });
 
  // ── write ───────────────────────────────────────────────────────────────────────
  it("POST /v1/channels/:channelId/messages — refuses, and writes nothing", async () => {
    attacked.add("POST /v1/channels/:channelId/messages");
    const verdict = await writeAttack(
      url,
      t.attacker.credential,
      {
        method: "POST",
        path: `/v1/channels/${t.victim.channelId}/messages`,
        body: { text: "from the attacker" },
      },
      {
        method: "POST",
        path: `/v1/channels/${ABSENT_UUID}/messages`,
        body: { text: "from the attacker" },
      },
      () => t.victim.repo.listMessages(t.victim.channelId, { limit: 50 }),
    );
    expect(verdict.differences, verdict.differences.join("; ")).toEqual([]);
    // THE STATE READ IS THE POINT: a 404 that completed the write is the case no
    // status code reveals.
    expect(verdict.stateChanged, "the victim's messages moved").toBe(false);
  });
 
  it("POST /internal/messages — a foreign channel_id refuses, and writes nothing", async () => {
    attacked.add("POST /internal/messages");
    const verdict = await writeAttack(
      url,
      attackerToken,
      {
        method: "POST",
        path: "/internal/messages",
        body: { channel_id: t.victim.channelId, text: "from the attacker" },
      },
      {
        method: "POST",
        path: "/internal/messages",
        body: { channel_id: ABSENT_UUID, text: "from the attacker" },
      },
      () => t.victim.repo.listMessages(t.victim.channelId, { limit: 50 }),
    );
    expect(verdict.differences, verdict.differences.join("; ")).toEqual([]);
    expect(verdict.stateChanged, "the victim's messages moved").toBe(false);
  });
 
  it("POST /internal/backfill — a foreign channel yields nothing", async () => {
    attacked.add("POST /internal/backfill");
    const verdict = await writeAttack(
      url,
      attackerToken,
      {
        method: "POST",
        path: "/internal/backfill",
        body: { cursors: { [t.victim.channelId]: 0 } },
      },
      { method: "POST", path: "/internal/backfill", body: { cursors: { [ABSENT_UUID]: 0 } } },
      () => t.victim.repo.listMessages(t.victim.channelId, { limit: 50 }),
    );
    expect(verdict.differences, verdict.differences.join("; ")).toEqual([]);
    expect(verdict.stateChanged).toBe(false);
    // AND THE BODY MUST CARRY NO ROWS. A backfill answers with messages rather than a
    // status, so "indistinguishable from absent" is necessary and not sufficient — an
    // endpoint that returned the victim's messages for BOTH requests would pass the
    // pair comparison and fail the product.
    const serialised = JSON.stringify(verdict.foreign.body ?? "");
    expect(serialised).not.toContain(t.victim.messageId);
    expect(serialised).not.toContain("victim says something");
  });
 
  // ── credential ──────────────────────────────────────────────────────────────────
  it("POST /auth/dev-token — a token minted for one environment is refused by another", async () => {
    attacked.add("POST /auth/dev-token");
    const verdict = await credentialAttack(
      url,
      t.attacker.credential,
      // The attacker asks for a token naming the VICTIM's user. Either the mint
      // refuses, or the token it hands back must be useless against the victim.
      t.victim.userExternalId,
      { method: "GET", path: `/v1/channels/${t.victim.channelId}/messages` },
    );
    if (verdict.minted) {
      expect(
        verdict.crossStatus,
        `a token minted with the attacker's key reached the victim's channel: ` +
          JSON.stringify(verdict.crossBody),
      ).not.toBe(200);
    }
  });
 
  it("POST /internal/session — the attacker's token resolves only its own environment", async () => {
    attacked.add("POST /internal/session");
    const res = await fetch(`${url}/internal/session`, {
      method: "POST",
      headers: { authorization: `Bearer ${attackerToken}` },
    });
    const body: unknown = res.ok ? await res.json() : null;
    const serialised = JSON.stringify(body ?? "");
    // Whatever it answers, nothing of the victim's may appear in it.
    expect(serialised).not.toContain(t.victim.userId);
    expect(serialised).not.toContain(t.victim.channelId);
    expect(serialised).not.toContain(t.victim.environmentId);
  });
 
  // ── and the suite accounts for itself ───────────────────────────────────────────
  it("ran an attack for every route the classification says to attack", () => {
    const shouldAttack = CLASSIFICATIONS.filter((c) => c.shape !== "exempt").map(targetKey);
    const missing = shouldAttack.filter((k) => !attacked.has(k));
    // A classification saying `write` with no attack written for it is the same hole
    // as a route with no classification, one level up. Named, because the useful half
    // of this failure is WHICH route nobody wrote a test for.
    expect(missing, `classified but never attacked: ${missing.join(", ")}`).toEqual([]);
  });
});

Ba attack, không phải bốn. Platform chưa có list route, vì vậy list attack sẽ là function không có target — chương thêm collection endpoint đầu tiên sẽ thêm shape và attack cùng lúc. Body comparison cũng không cần exclude gì: envelope gồm code, message, docs_url; cả ba phải khớp. Khi per-request field được thêm, chương thêm field đó phải lập luận vì sao bỏ nó tại đây.

Suite chạy một attack trên mỗi route không exempt rồi kiểm tra rằng nó thực sự đã chạy. Test cuối áp cùng ý tưởng ở tầng cao hơn. Classification nói write nhưng không có attack là cùng một lỗ hổng với route không classification.

Leak chưa có endpoint

Gauntlet attack endpoint. Còn câu hỏi thứ hai nó không thể đặt: table không có path quay về environment sẽ bị lộ khi query đầu tiên join table đó, và trước thời điểm ấy không gì fail.

Ta đọc từ information_schema thay vì schema.ts, vì table do migration thêm mà Drizzle không model sẽ vô hình với check đọc model — đúng loại table check này tồn tại để bắt.

services/api/src/db/catalogue.ts
import { sql } from "drizzle-orm";
 
import type { Db } from "./client";
 
// WHERE EVERY TABLE KEEPS ITS TENANT (FR-012, constitution I).
//
// The gauntlet attacks endpoints. This asks the other half of the question: a table
// that carries no path back to an environment is A LEAK WITH NO ENDPOINT YET — nothing
// is exposing it today, and the first query that joins it will.
//
// DERIVED FROM `information_schema`, NOT FROM `schema.ts`, because the database is what
// the queries run against. A table added by a migration and never modelled in Drizzle is
// invisible to a check that reads the model, and that is exactly the table this exists
// to catch.
//
// It lives here rather than in the test that calls it because this directory is the only
// place the lint ban permits `drizzle-orm` (constitution I, ADR-16). A catalogue query
// written inline in the test would need an exemption for as long as it lived.
 
/** How a row in this table is traced back to one environment. */
export type TenantPath = "direct" | "hop" | "spine";
 
export interface TableClassification {
  table: string;
  /** `null` means the table matches none of the three, which fails the check. */
  path: TenantPath | null;
  /** For `hop`: the `direct` tables its foreign keys reach. */
  via: string[];
  /** For `spine`: why it has no tenant column. */
  reason?: string;
}
 
// THE SPINE, AS A LIST WITH A REASON EACH AND NOT A PATTERN.
//
// A pattern silently absorbs the next table that happens to match it, which is the
// opposite of what this check is for. Adding a table here should be an edit somebody
// has to justify in writing, and the reason is stored so the justification outlives
// the person who made it.
//
// These six are tenancy itself — the tables an environment_id would point INTO — plus
// the migration ledger, which predates all of it.
const SPINE: ReadonlyArray<readonly [string, string]> = [
  ["organisations", "the root of the tenancy tree; nothing is above it to scope to"],
  ["applications", "belongs to an organisation, which is the scope"],
  ["environments", "IS the scope — an environment_id here would point at itself"],
  [
    "humans",
    "a person, not a tenant's record; one human may belong to several organisations",
  ],
  ["memberships", "joins humans to organisations, above the environment level"],
  [
    "schema_migrations",
    "the migration ledger; it predates tenancy and belongs to the database",
  ],
];
 
/** The spine, for anyone who needs to state it rather than derive it. */
export const SPINE_TABLES: readonly string[] = SPINE.map(([t]) => t);
 
export interface CatalogueRow extends Record<string, unknown> {
  table_name: string;
  has_environment_id: boolean;
  fk_targets: string[] | null;
}
 
/** Every base table in `public`, each classified into exactly one of the three paths —
 * or into none, which is the answer that fails a build. */
export async function classifyTables(db: Db): Promise<TableClassification[]> {
  const rows = (
    await db.execute<CatalogueRow>(sql`
      WITH base AS (
        SELECT table_name
        FROM information_schema.tables
        WHERE table_schema = 'public' AND table_type = 'BASE TABLE'
      ),
      direct AS (
        SELECT table_name
        FROM information_schema.columns
        WHERE table_schema = 'public' AND column_name = 'environment_id'
      )
      SELECT
        b.table_name,
        (b.table_name IN (SELECT table_name FROM direct)) AS has_environment_id,
        (
          -- ::text is load-bearing. information_schema columns are sql_identifier,
          -- and node-pg has no parser for an array of them: the row arrives as the
          -- literal string {channels,users} and iterating it yields a brace, which
          -- is how this was found rather than reasoned about.
          SELECT array_agg(DISTINCT ccu.table_name::text)
          FROM information_schema.table_constraints tc
          JOIN information_schema.constraint_column_usage ccu
            ON ccu.constraint_name = tc.constraint_name
           AND ccu.table_schema = tc.table_schema
          WHERE tc.constraint_type = 'FOREIGN KEY'
            AND tc.table_schema = 'public'
            AND tc.table_name = b.table_name
            AND ccu.table_name IN (SELECT table_name FROM direct)
        ) AS fk_targets
      FROM base b
      ORDER BY b.table_name
    `)
  ).rows;
 
  return rows.map(classifyRow);
}
 
/** THE CLASSIFICATION, SEPARATED FROM THE QUERY, and separated for a reason worth
 * stating: the interesting arm is the one that returns `null`, and it cannot execute
 * against a real database that has no unclassified table — which is exactly the state
 * this check exists to keep. So the branch that fires only when somebody adds a table
 * is the one branch a live run can never reach.
 *
 * Pure, so a unit test can drive all four arms with rows it makes up. A file with
 * nothing to mock has no reason to be partially tested. */
export function classifyRow(row: CatalogueRow): TableClassification {
  const via = row.fk_targets ?? [];
  // ORDER MATTERS, AND ONLY IN ONE PLACE: a spine table with no environment_id and no
  // foreign key classifies the same either way, but checking `direct` first means a
  // future spine table that GAINS the column reports as `direct` and its list entry
  // becomes visibly wrong rather than silently ignored.
  if (row.has_environment_id) return { table: row.table_name, path: "direct", via };
  if (via.length > 0) return { table: row.table_name, path: "hop", via };
  const reason = new Map(SPINE).get(row.table_name);
  if (reason !== undefined) return { table: row.table_name, path: "spine", via, reason };
  return { table: row.table_name, path: null, via };
}
services/api/src/isolation/tenant-scope.itest.ts
import { afterAll, beforeAll, describe, expect, it } from "vitest";
 
import { classifyTables, SPINE_TABLES, type TableClassification } from "../db/catalogue";
import { createDb, createPool } from "../db/client";
 
import type { Db } from "../db/client";
 
// THE STRUCTURAL HALF (FR-012, constitution I).
//
// The gauntlet attacks the endpoints that exist. This asks about THE LEAK THAT HAS NO
// ENDPOINT YET: a table with no path back to an environment is exposed by the first
// query that joins it, and nothing before that moment fails.
//
// WHAT THIS DOES NOT CHECK, and the distinction is easy to lose: that every table HAS a
// tenant path, not that every QUERY respects the one it has. The gauntlet makes the
// second claim, endpoint by endpoint. Neither implies the other.
 
describe("every table has a path to one tenant", () => {
  // `ReturnType` rather than `import type pg from "pg"`: the driver's own types are
  // behind the same ban as the driver, and a type-only import is still an import to
  // `no-restricted-imports`.
  let pool: ReturnType<typeof createPool>;
  let db: Db;
  let tables: TableClassification[];
 
  beforeAll(async () => {
    pool = createPool();
    db = createDb(pool);
    tables = await classifyTables(db);
  }, 30_000);
 
  afterAll(async () => {
    await pool?.end();
  });
 
  it("classifies every base table as direct, hop or spine", () => {
    const unclassified = tables.filter((t) => t.path === null).map((t) => t.table);
    // The message names the tables, because the useful half of this failure is WHICH
    // table appeared — a new migration's, almost always.
    expect(
      unclassified,
      `these tables have no path to an environment: ${unclassified.join(", ")}. ` +
        `Add environment_id, add a foreign key to a table that has one, or add it to ` +
        `SPINE in db/catalogue.ts with a reason.`,
    ).toEqual([]);
  });
 
  it("classifies each table exactly once", () => {
    const seen = new Set<string>();
    for (const t of tables) {
      expect(seen.has(t.table), `${t.table} classified twice`).toBe(false);
      seen.add(t.table);
    }
    expect(seen.size).toBe(tables.length);
  });
 
  it("has no spine entry for a table that does not exist", () => {
    // THE DIRECTION THAT CATCHES A DROPPED TABLE. A spine entry outliving its table is
    // an exemption standing over nothing — harmless today, and waiting to cover a
    // future table that happens to reuse the name.
    const present = new Set(tables.map((t) => t.table));
    const stale = SPINE_TABLES.filter((t) => !present.has(t));
    expect(stale, `spine names tables that do not exist: ${stale.join(", ")}`).toEqual([]);
  });
 
  it("gives every spine table a reason", () => {
    const reasonless = tables
      .filter((t) => t.path === "spine" && (t.reason ?? "").trim() === "")
      .map((t) => t.table);
    expect(reasonless).toEqual([]);
  });
 
  it("reports the shape of the schema", () => {
    const by = (p: TableClassification["path"]) => tables.filter((t) => t.path === p);
    const direct = by("direct");
    const hop = by("hop");
    const spine = by("spine");
    console.log(
      `tenant paths: ${tables.length} tables — ${direct.length} direct, ` +
        `${hop.length} hop, ${spine.length} spine`,
    );
    expect(direct.length + hop.length + spine.length).toBe(tables.length);
    // A hop with no target is a hop in name only, and the query that produced it
    // would have to be wrong for this to happen — which is why it is asserted.
    for (const t of hop) {
      expect(t.via.length, `${t.table} is a hop to nowhere`).toBeGreaterThan(0);
    }
  });
});
what the schema looks like from here
tenant paths: 11 tables — 3 direct, 2 hop, 6 spine

Socket không nằm trong router

WebSocket không xuất hiện trong router.stack, nên toàn bộ phần trên không nhìn thấy nó. Socket có nửa riêng; attack surface cũng được suy ra theo cùng cách, từ frame union của protocol.

API chạy như child process với PORT=0 rồi báo port thật đã bind. Fixed port race với sibling suite cũng bind nó; child từ lần chạy trước còn giữ port khiến health check thành công dù service đó chưa từng nghe data của lần chạy này — ba assertion trông không liên quan nhưng chung một fixture. Để operating system chọn port loại bỏ table mapping cần duy trì.

services/gateway/src/isolation-fixtures.ts
import { spawn, type ChildProcess } from "node:child_process";
import { randomUUID } from "node:crypto";
import { createRequire } from "node:module";
import { dirname, join } from "node:path";
import { fileURLToPath } from "node:url";
 
/** Two tenants and a running api, for the socket half of the gauntlet.
 *
 * SEEDING GOES THROUGH THE API'S OWN REPOSITORY, imported from its build output — the
 * test-only seam chapter 2.8 established, for the same reason: there is no admin API for
 * environments or keys, and inventing one for a test would be inventing product. The
 * gateway does not depend on the api package and must not start.
 *
 * THE API RUNS AS A CHILD AND THE GATEWAY IN PROCESS. In process, because a test that
 * cannot reach the gateway's own state cannot check what it subscribed to; as a child,
 * because importing the api would make this service depend on the api's framework to
 * test itself, and not knowing how the api is built is the whole of ADR-05. */
 
const HERE = dirname(fileURLToPath(import.meta.url));
const REPO = join(HERE, "..", "..", "..");
const require_ = createRequire(import.meta.url);
 
interface Seeder {
  createEnvironment: (db: unknown, input: { name: string }) => Promise<{ id: string }>;
  createApiKey: (
    db: unknown,
    input: { environmentId: string },
  ) => Promise<{ credential: string }>;
  Repository: new (
    db: unknown,
    environmentId: string,
  ) => {
    createUser: (externalId: string, name?: string) => Promise<{ id: string }>;
    createChannel: (
      externalId: string,
      type: string,
      name?: string,
    ) => Promise<{ id: string }>;
    addMember: (channelId: string, userId: string) => Promise<boolean>;
    sendMessage: (
      channelId: string,
      input: { text: string; userId?: string },
    ) => Promise<{ id: string }>;
  };
}
 
export interface SocketTenant {
  environmentId: string;
  credential: string;
  userExternalId: string;
  userId: string;
  channelId: string;
}
 
export interface SocketTenants {
  /** The caller. Its token is the one every attack presents. */
  attacker: SocketTenant;
  /** The tenant whose identifiers the attacker borrows. */
  victim: SocketTenant;
  apiUrl: string;
  stop: () => void;
}
 
/** THE PORT COMES FROM THE CHILD, NOT FROM A TABLE.
 *
 * The api is started with `PORT=0` and reports the port it bound. A hand-allocated band
 * per suite is a table nothing checks: two suites eventually overlap, or a band grows to
 * contain a port the lane itself runs, and the failure is a health check that succeeds
 * against the wrong service. Asking the operating system removes the table. */
async function startApi(): Promise<{ url: string; stop: () => void }> {
  const dist = join(REPO, "services", "api", "dist");
  const child: ChildProcess = spawn("node", [join(dist, "main.js")], {
    env: { ...process.env, PORT: "0" },
    stdio: ["ignore", "pipe", "pipe"],
  });
  const port = await new Promise<number>((resolve, reject) => {
    const timer = setTimeout(() => reject(new Error("api never reported a port")), 30_000);
    let buffered = "";
    child.stdout?.on("data", (chunk: Buffer) => {
      buffered += chunk.toString();
      for (const line of buffered.split("\n")) {
        if (!line.trim()) continue;
        try {
          const parsed = JSON.parse(line) as { msg?: string; port?: number };
          if (parsed.msg === "listening" && typeof parsed.port === "number") {
            clearTimeout(timer);
            resolve(parsed.port);
            return;
          }
        } catch {
          /* a partial line; the next chunk completes it */
        }
      }
    });
    child.on("exit", (code) => {
      clearTimeout(timer);
      reject(new Error(`api exited before listening (code ${String(code)})`));
    });
  });
  return { url: `http://127.0.0.1:${port}`, stop: () => child.kill() };
}
 
export async function seedSocketTenants(): Promise<SocketTenants> {
  const dist = join(REPO, "services", "api", "dist");
  const client = require_(join(dist, "db", "client.js")) as {
    createDb: (pool: unknown) => unknown;
    createPool: () => unknown;
  };
  const seeder = require_(join(dist, "db", "repository.js")) as Seeder;
  const db = client.createDb(client.createPool());
 
  const seed = async (label: string): Promise<SocketTenant> => {
    const environment = await seeder.createEnvironment(db, {
      name: `socket-isolation-${label}-${randomUUID().slice(0, 8)}`,
    });
    const repo = new seeder.Repository(db, environment.id);
    const userExternalId = `${label}-user`;
    const user = await repo.createUser(userExternalId, `${label} user`);
    const channel = await repo.createChannel(`${label}-channel`, "public");
    await repo.addMember(channel.id, user.id);
    await repo.sendMessage(channel.id, { text: `${label} says something`, userId: user.id });
    const key = await seeder.createApiKey(db, { environmentId: environment.id });
    return {
      environmentId: environment.id,
      credential: key.credential,
      userExternalId,
      userId: user.id,
      channelId: channel.id,
    };
  };
 
  const attacker = await seed("attacker");
  const victim = await seed("victim");
  const api = await startApi();
  return { attacker, victim, apiUrl: api.url, stop: api.stop };
}
 
/** A token for one tenant's user, minted with that tenant's key. */
export async function mintToken(
  apiUrl: string,
  credential: string,
  user: string,
): Promise<string> {
  const res = await fetch(`${apiUrl}/auth/dev-token`, {
    method: "POST",
    headers: { authorization: `Bearer ${credential}`, "content-type": "application/json" },
    body: JSON.stringify({ user }),
  });
  if (!res.ok) throw new Error(`dev-token: ${res.status}`);
  return ((await res.json()) as { token: string }).token;
}
services/gateway/src/isolation.itest.ts
import { randomUUID } from "node:crypto";
import type { Server } from "node:http";
import type { AddressInfo } from "node:net";
 
import { docsUrl, frameSchema } from "@relay/protocol";
import { createLogger, serve, type Logger } from "@relay/service-kit";
import { WebSocket } from "ws";
import { afterAll, beforeAll, describe, expect, it } from "vitest";
 
import { createApiClient } from "./api-client.js";
import { mintToken, seedSocketTenants, type SocketTenants } from "./isolation-fixtures.js";
import { attachSessions } from "./session.js";
 
// THE SOCKET HALF OF THE GAUNTLET (FR-007, NFR-SEC-09, constitution I).
//
// The api's gauntlet attacks every HTTP route. None of it reaches here: A WEBSOCKET IS
// NOT IN `router.stack`, so the derived target list cannot see it, and this file is the
// only place the socket gets attacked with another tenant's identifiers.
//
// The list of things to attack is derived here too, from `frameSchema`'s own members
// rather than from a list somebody typed — same argument as the router, one protocol
// down. A frame type added to the union and forgotten here is exactly the case that
// would otherwise ship unattacked.
 
const silent: Logger = createLogger("gateway", () => {});
 
/** Every frame type the protocol declares, read off the union.
 *
 * ASSERTED NON-EMPTY BEFORE IT IS USED, for the reason the router derivation records: a
 * shape change in zod that made this return nothing would leave the suite green while
 * it attacked no frames at all. */
function declaredFrameTypes(): string[] {
  const options = (frameSchema as unknown as { options?: unknown[] }).options ?? [];
  const types: string[] = [];
  for (const option of options) {
    const shape = (option as { shape?: { type?: { value?: unknown } } }).shape;
    const value = shape?.type?.value;
    if (typeof value === "string") types.push(value);
  }
  return types;
}
 
async function closeCode(socket: WebSocket): Promise<number> {
  return new Promise((resolve, reject) => {
    socket.on("close", (code) => resolve(code));
    socket.on("error", () => undefined);
    setTimeout(() => reject(new Error("no close within 5s")), 5_000);
  });
}
 
async function firstFrame(socket: WebSocket, type: string): Promise<Record<string, unknown>> {
  return new Promise((resolve, reject) => {
    socket.on("message", (raw) => {
      const frame = JSON.parse(raw.toString()) as Record<string, unknown>;
      if (frame.type === type) resolve(frame);
    });
    socket.on("close", (code) => reject(new Error(`closed ${code}`)));
    setTimeout(() => reject(new Error(`no ${type} within 5s`)), 5_000);
  });
}
 
describe("the socket refuses another tenant's identifiers", () => {
  let t: SocketTenants;
  let server: Server;
  let url: string;
  let attackerToken: string;
 
  beforeAll(async () => {
    t = await seedSocketTenants();
    server = serve({
      service: "gateway",
      health: () => ({}),
      logger: silent,
      notFoundDocsUrl: docsUrl("not_found"),
    });
    attachSessions({ server, api: createApiClient(t.apiUrl), logger: silent });
    await new Promise<void>((resolve) => server.listen(0, resolve));
    url = `ws://127.0.0.1:${(server.address() as AddressInfo).port}`;
    attackerToken = await mintToken(t.apiUrl, t.attacker.credential, t.attacker.userExternalId);
  }, 90_000);
 
  afterAll(async () => {
    await new Promise<void>((resolve) => server?.close(() => resolve()));
    t?.stop();
  });
 
  it("derives the frame types from the protocol, and finds some", () => {
    const types = declaredFrameTypes();
    // An empty derivation is a broken derivation, not a small protocol.
    expect(types.length).toBeGreaterThan(1);
    expect(types).toContain("message.send");
  });
 
  it("a connection ack names nothing belonging to the other tenant", async () => {
    const socket = new WebSocket(`${url}/v1/ws?token=${attackerToken}`);
    const ack = await firstFrame(socket, "connection.ack");
    const serialised = JSON.stringify(ack);
    expect(serialised).not.toContain(t.victim.channelId);
    expect(serialised).not.toContain(t.victim.environmentId);
    expect(serialised).not.toContain(t.victim.userId);
    socket.close();
  }, 20_000);
 
  it("message.send to the other tenant's channel is refused", async () => {
    const socket = new WebSocket(`${url}/v1/ws?token=${attackerToken}`);
    await firstFrame(socket, "connection.ack");
    socket.send(
      JSON.stringify({
        type: "message.send",
        payload: {
          idem_key: randomUUID(),
          channel: t.victim.channelId,
          text: "from the attacker",
        },
      }),
    );
    const error = await firstFrame(socket, "error");
    const payload = error.payload as { code?: string; message?: string };
    // The refusal must not name what it refused. An error that echoes the channel id
    // back tells the attacker the channel exists, which is the leak the HTTP gauntlet
    // proves is absent on every route — the socket does not get an exemption.
    expect(JSON.stringify(payload)).not.toContain(t.victim.channelId);
    expect(payload.code).toBeTruthy();
    socket.close();
  }, 20_000);
 
  it("every declared frame type that is not message.send is refused inbound", async () => {
    // SCHEMA VALIDATION RUNS BEFORE THE TYPE CHECK, and that shapes what this can
    // claim. A frame whose payload does not match its own schema is answered
    // `invalid_frame` and never reaches the rule that says clients may not utter a
    // server frame — so a loop sending `{}` for every type would pass while testing
    // the parser, not the rule. The first draft of this test did exactly that.
    //
    // So: every declared type must be refused SOMEHOW, and one well-formed server
    // frame must be refused BY THE RULE.
    const inboundOnly = declaredFrameTypes().filter((type) => type !== "message.send");
    expect(inboundOnly.length).toBeGreaterThan(0);
    for (const type of inboundOnly) {
      const socket = new WebSocket(`${url}/v1/ws?token=${attackerToken}`);
      await firstFrame(socket, "connection.ack");
      socket.send(JSON.stringify({ type, payload: {} }));
      const error = await firstFrame(socket, "error");
      expect((error.payload as { code?: string }).code, `${type} was not refused`).toBeTruthy();
      socket.close();
    }
 
    // `message.ack` carries `{ seq }`, which is the easiest valid server frame to
    // build — so it is the one that proves the rule rather than the parser.
    const socket = new WebSocket(`${url}/v1/ws?token=${attackerToken}`);
    await firstFrame(socket, "connection.ack");
    socket.send(JSON.stringify({ type: "message.ack", payload: { seq: 1 } }));
    const error = await firstFrame(socket, "error");
    expect((error.payload as { code?: string }).code).toBe("unknown_frame_type");
    // A protocol violation closes the connection (EIR-WS-06's 4002).
    await expect(closeCode(socket)).resolves.toBe(4002);
  }, 60_000);
 
  it("a token minted by one tenant cannot open a session for the other", async () => {
    // The attacker asks its OWN api for a token naming the victim's user. Either the
    // mint refuses, or the session it opens must resolve nothing of the victim's.
    const borrowed = await mintToken(
      t.apiUrl,
      t.attacker.credential,
      t.victim.userExternalId,
    ).catch(() => null); // refused at the mint is the stronger answer
    if (borrowed === null) return;
    const socket = new WebSocket(`${url}/v1/ws?token=${borrowed}`);
    try {
      const ack = await firstFrame(socket, "connection.ack");
      expect(JSON.stringify(ack)).not.toContain(t.victim.channelId);
    } catch {
      // A closed socket is a refusal, which is also correct.
      await expect(closeCode(socket)).resolves.toBeGreaterThan(0);
    }
    socket.close();
  }, 20_000);
});

Hai test pass khi chưa tấn công gì

Cả hai socket test phía trên đều xanh trước khi đúng; cách chúng sai đáng giá hơn cách chúng đúng.

Cross-tenant send dùng payload { channel_id, text }. Frame thật mang { idem_key, channel, text }, nên socket trả invalid_frame — "expected string, received undefined" và từ chối vì malformed trước khi tenant check chạy. Test assert reply là refusal và không chứa channel victim. Cả hai đều đúng nhưng không liên quan isolation.

Frame-union loop cũng vậy. Schema validation chạy trước type check, nên gửi {} cho mọi server frame type nhận invalid_frame và không bao giờ tới rule cấm client phát server frame. Loop chỉ đang test parser.

Fix cho trường hợp hai không phải xoá loop — mọi declared type vẫn phải bị từ chối bằng cách nào đó — mà là thêm một server frame well-formed, message.ack với { seq }, đủ để tới rule và chứng minh nó.

Blast radius, từng file một

Ba thay đổi chương này cần nhưng không dự định từ đầu.

Các integration suite không thể chạy cạnh nhau. Ba trong chín API suite fail với duplicate key value violates unique constraint "pg_type_typname_nsp_index" — migration đồng thời race để tạo cùng type. Ba trên ba concurrent run fail; hai trên hai serialized run pass. Đây là lỗi tái hiện được, không phải flaky.

API cũng log port nó yêu cầu, tức 0, thay vì port thực tế.

Cuối cùng, seed hai tenant làm hỏng test hàng xóm. signup.itest.ts đếm toàn bộ organisations trước và sau failed provision rồi assert hai số bằng nhau — global claim về local operation. Failure expected 452 to be 451 chẳng nói gì về neighbour. Test ngay phía trên đã scope câu hỏi bằng id.

package.json
@@ -9,13 +9,13 @@
   "scripts": {
     "dev": "turbo run dev",
     "lint": "turbo run //#lint:root",
     "lint:root": "eslint .",
     "typecheck": "turbo run typecheck",
     "test": "turbo run test",
-    "test:integration": "turbo run test:integration",
+    "test:integration": "turbo run test:integration --concurrency=1",
     "build": "turbo run build"
   },
   "devDependencies": {
     "@eslint/js": "^10.0.1",
     "@types/node": "^26.1.2",
     "eslint": "^10.8.0",
services/api/src/main.ts
@@ -7,12 +7,25 @@ import { AppModule } from "./app.module";
 
 // Nest's own banner logger stays off: this workspace already decided what a
 // log line looks like (one JSON object, NFR-OBS-01), and the framework does
 // not get a second opinion.
 async function bootstrap(): Promise<void> {
   const app = await NestFactory.create(AppModule, { logger: false });
-  const port = Number(process.env.PORT ?? 4000);
-  await app.listen(port);
+  const requested = Number(process.env.PORT ?? 4000);
+  await app.listen(requested);
+  // THE PORT IT GOT, NOT THE PORT IT ASKED FOR.
+  //
+  // `PORT=0` asks the operating system for any free port, which is what a test
+  // spawning this service should do — a fixed port races whichever sibling suite also
+  // binds one, and a previous run's child still holding it makes a health check succeed
+  // against a service that has never heard of this run's data. Three unrelated-looking
+  // assertions, one fixture.
+  //
+  // But a parent can only use the number if this process reports it, and logging
+  // `requested` prints 0. So the bound address is read back and logged.
+  const address = app.getHttpServer().address() as { port?: number } | string | null;
+  const port =
+    typeof address === "object" && address !== null ? (address.port ?? requested) : requested;
   createLogger("api").log("info", "listening", { port });
 }
 
 void bootstrap();
services/api/src/tenancy/signup.itest.ts
@@ -149,37 +149,35 @@ describe("signup", () => {
     expect(Number(counts.apps)).toBe(1);
     expect(Number(counts.envs)).toBe(1);
     expect(Number(counts.owners)).toBe(1);
   });
 
   it("writes the full set or nothing when provisioning fails (invariant 1)", async () => {
-    const before = await db.execute(
-      `SELECT count(*)::int AS n FROM organisations`,
-    );
-    // Force a failure inside the transaction, after the organisation insert:
-    // an organisation name that is fine and a provider value the CHECK
-    // constraint refuses.
+    // SCOPED TO THE ROW THIS TEST CREATES, not to the table.
+    //
+    // This read `SELECT count(*) FROM organisations` before and after and asserted the
+    // two matched. That is a GLOBAL claim about a LOCAL operation: any other suite that
+    // provisions a tenant between the two reads breaks it, and the failure —
+    // `expected 452 to be 451` — says nothing about a neighbour. The isolation
+    // fixtures seed two tenants and did exactly that.
+    //
+    // What invariant 1 actually claims is that the failed transaction left NOTHING
+    // behind. That is a question about one organisation, and the test above already
+    // asks its questions that way.
     await expect(
       provisionOrganisation(db, {
         provider: "not-a-provider",
         providerAccountId: "x",
         organisationName: "doomed org",
       }),
     ).rejects.toThrow();
-    const after = await db.execute(
-      `SELECT count(*)::int AS n FROM organisations`,
-    );
-    // Nothing survived — no half-built tenant, which is the whole point of the
-    // single transaction.
-    expect((after.rows[0] as { n: number }).n).toBe(
-      (before.rows[0] as { n: number }).n,
-    );
-    const orphan = await db.execute(
+    const doomed = await db.execute(
       `SELECT count(*)::int AS n FROM organisations WHERE name = 'doomed org'`,
     );
-    expect((orphan.rows[0] as { n: number }).n).toBe(0);
+    // No half-built tenant, which is the whole point of the single transaction.
+    expect((doomed.rows[0] as { n: number }).n).toBe(0);
   });
 
   it("recognises a returning owner instead of creating a second organisation (invariant 2)", async () => {
     const accountId = `returning-${Date.now()}`;
     const { first, second } = await withFreshIdentity(accountId, async () => ({
       first: await signUp("code-a"),

Những điều chương này chủ ý để lại cho sau

Cross-tenant send qua socket nhận internal_error: "send failed". Isolation vẫn giữ — nó không phân biệt được với failure khác — nhưng code sai và client không thể hành động. Đó là refusal mặc quần áo của error; chương ghi lại thay vì sửa.

Suite không có unit test riêng. classifyRow trả null cho table chưa classify; comparePair báo difference; không arm nào có thể chạy trên lane khoẻ vì trạng thái chúng mô tả chính là thứ chương này ngăn. Instrument chưa từng fire là instrument chưa được test — đó là một chương riêng.