From app to infrastructure — finding the real product

We are going to start this series the way most real projects start: with an idea that is slightly wrong.

The idea is this. We keep watching product teams add chat to their applications — a support thread here, a buyer-and-seller conversation there — and we keep watching it go badly. So we decide to build a chat application: a clean, fast, well-designed messaging product. It seems like a good idea for about a day. By the end of this chapter we abandon it, on purpose, in favor of a much better one — and the process of abandoning it produces the two artifacts every serious project needs before any code exists: a positioning statement and a list of non-goals.

The two-week feature

Every builder we might sell to has had this conversation. A product manager says the customers are asking for messaging. An engineer looks at the problem and sees a WebSocket server and a messages table. Two weeks, maybe three. And the striking thing is that the engineer is right — the naïve build really does work. In development. For a while.

Then it meets reality, and each innocent phrase in the estimate unfolds into a system. Let's walk the unfolding, because the gap between the phrase and the system is where our product hides.

"Just open a WebSocket." One server, one socket, no problem. But the moment we run two server instances behind a load balancer, the sender's connection lives on one instance and the recipient's on another. Now we need connection state shared across instances, sticky routing or a pub/sub fabric between them, and — because phones go through tunnels — reconnection with backfill, so a dropped client can ask "what did I miss?" and get a correct answer.

"Store the messages." A messages table stores messages fine. What it does not do by itself is guarantee ordering under concurrent sends, make writes idempotent so a retried request doesn't duplicate a message, paginate a hot table with cursors instead of collapsing offsets, or enforce a retention policy when legal asks how long we keep customer conversations.

"Show who's online." Presence looks like a boolean. It is actually a heartbeat, a time-to-live, and a story for what happens during a network partition — because "green dot that lies" is worse than no green dot.

"Send a notification." Behind one notification sits a delivery state machine: fan-out to users who are offline, retry semantics for the ones we could not reach, and deduplication so retries don't become double notifications.

"Let support look at a thread." The moment a human moderator enters the picture we owe them moderation tooling, an audit trail, export, and deletion on request — the compliance surface nobody scoped.

"How much is this costing us?" Finance eventually asks for usage per customer. Answering means metering millions of events per tenant, and the operational database — the one serving live chat — is exactly the wrong place to run that query.

So the two-week feature is a six-month system. But here is the observation that matters more, and it is the emotional center of this chapter: the cost is not the first version. The cost is the second year. Someone now carries a pager for a system that is not the company's product. Some engineer now owns messaging permanently. Every team that builds chat in-house signs up for that tax, and most of them don't know it yet.

Who already solved this — and the gap they left

We are not the first to notice this. So before we build anything, we survey who a team in that conversation can turn to. There are three doors.

Door one: build it anyway. Full control, perfectly tailored — and the permanent maintenance tax we just described, paid on a feature that does not differentiate the product. For most teams this is the default, not a decision.

Door two: the incumbent platforms. Stream, Sendbird, PubNub, Ably. These are capable products, and we should say so honestly — competing with them on feature breadth would be foolish. But they are priced and scoped for large deployments, their surface areas are broad enough to take real time to learn, and their usage reporting is opaque enough that forecasting next quarter's bill is hard. A fifteen-engineer team evaluating them feels like they are buying an airliner to cross a river.

Door three: general real-time infrastructure. Pusher, Supabase Realtime, plain Redis pub/sub. These give us a transport — bytes move from here to there, quickly. But a transport is not a chat: channels, membership, history, and read state are still ours to build. Door three is door one with the easiest part outsourced.

Look at the shape of what's missing. Between "a transport that knows nothing about chat" and "a platform too big to hold in your head" there is a gap: a real chat domain model — channels, membership, history, delivery, read state — with a surface area small enough to understand in an afternoon. That gap is a product.

The product hiding in the failure

And now our original idea quietly dies. Notice what we've been describing all along: not screens, not emoji pickers, not a beautiful thread view — connection fabrics, ordering guarantees, delivery state machines, metering pipelines. The hard, valuable thing in this space is not another chat application. It is the infrastructure underneath every chat feature: the part every team rebuilds badly, the part none of their customers ever sees.

So we pivot the idea: we are not building a chat app. We are building the chat infrastructure that product teams embed in their apps — an API and an SDK, the way Stripe is an API for payments and Twilio is an API for SMS. In five years, we want a team that needs in-app conversation to ship it in a week, not a quarter, with exactly that Stripe-level confidence.

This is the chapter's first lesson in product thinking, and it generalizes: the real product is often one level below the visible idea. The visible idea is what users describe. The product is what builders keep failing to build.

Not everyone. Someone.

"Chat infrastructure for everyone who needs chat" is not a product either — it is a prayer. A product needs a wedge: the specific customer for whom it is obviously, urgently right. The vision document commits to one.

The wedge is B2B SaaS products with a collaboration surface. Project management tools. Healthcare coordination platforms. Logistics dashboards. Professional services marketplaces. Teams of roughly five to fifty engineers, with no dedicated infrastructure group.

Why them? Because they share three traits, and each trait does real work in the argument:

1. Chat is adjacent to their core value, not the core itself. They will never staff a messaging team; building is a distraction they would love to refuse.
2. Their users are already identified and authenticated. We never have to become an identity provider — their backend vouches for who is who.
3. Their volume is meaningful but not internet-scale. Real load, real revenue — without the hundred-million-connection problems that belong to social networks.

Secondary segments — marketplaces needing buyer-seller conversations with moderation and audit trails, early-stage consumer apps where speed beats unit cost, internal platform teams standardizing messaging — fit the same shape. And some customers we name specifically so we can decline them: social networks at hundreds of millions of connections, regulated deployments demanding end-to-end encryption or on-premise installs, and teams who actually want a finished Slack, not infrastructure.

Saying it in one sentence

Everything so far now has to compress into a form we can be judged on. The classic positioning template forces every hard choice into one breath:

For [the target] who [the need], [the product] is a [the category] that [the benefit]. Unlike [the alternative], our product [the differentiator].

Here is Relay's, filled in — and we should read it slowly, because each clause is one of this chapter's sections compressed to a phrase:

For product engineering teams at B2B and marketplace software companies who need in-app messaging but cannot justify building and operating it, Relay is a chat infrastructure platform that provides real-time messaging as a hosted API with a client SDK, webhooks, and per-tenant usage analytics. Unlike building on raw WebSockets or adopting a heavyweight incumbent, Relay is API-first, transparent about delivery and usage, and small enough to understand in an afternoon.

The template earns its keep by what it refuses to allow. "For everyone" doesn't fit in the first slot. Naming the alternative honestly — including the incumbents we respect — forces the differentiator to be real: not "better chat," but API-first, transparent about usage, small enough to learn in an afternoon. If a claimed differentiator would not survive being read aloud to a customer of the named alternative, it comes out.

Non-goals are commitments, not suggestions

The positioning statement says what we are. Its mirror twin says what we are not — and in our experience, the mirror does more work. Scope creep never arrives as a bad idea; it arrives as a good idea that is not ours. The only defense is a written list of good ideas we are refusing, each with a reason attached. Relay's:

• Not a chat application. No hosted, brandable end-user product. This is the original idea of this chapter, formally buried.
• No end-to-end encryption in v1. Not because it is unfashionable — because it is incompatible with server-side moderation and history that our wedge market requires, and that market is not asking for E2E yet. A non-goal with a mechanism behind it is far harder to relitigate than a preference.
• Not an identity provider. Customers own their user directory; their backend vouches for their users. Refusing this scope is what makes our security story simple.
• No voice or video calls. Real-time A/V is a different transport problem with different economics — a whole other company, frankly. Media files — image, audio, and video attachments — are in: Relay hosts them, scans them, and meters their storage. The line is drawn at streams, not bytes.
• No AI copilot features beyond optional content classification. The temptation is strong, which is precisely why it is on the list.
• No native mobile SDKs in v1. One well-made JavaScript SDK that works in browsers, Node, and React Native beats four half-finished ones.

Notice the pattern: every entry names a real product someone will someday ask us to be, and gives the reason we decline. That reason is what the list is for. A year from now, "should we add threads?" is not a debate — it is a lookup.

The reasons buy us one more thing: a principled way to change our mind. That media clause above is a reversal — an earlier draft excluded hosted file storage outright, for stated reasons: storage cost, virus scanning, CDN concerns. Then a design answered every reason — uploads that travel straight to storage without touching our servers, scanning as an asynchronous pipeline — and the wedge market supplied the motive: a driver's voice note or a damaged-parcel photo is the message. So the non-goal was reversed out loud, reasons first. A non-goal without a reason can only be defended or abandoned; a non-goal with a reason can be honestly re-litigated.

Your turn

Now we produce the chapter's artifacts — for your own project idea, not ours. If you don't have one in your pocket, borrow this: a scheduling tool for veterinary clinics is adding reminders and a client portal. Twenty minutes of honest work beats two hours of polishing.

Exercise 1 — the positioning statement. Take the template:

For [target] who [need], [product] is a [category] that [benefit]. Unlike [alternative], our product [differentiator].

Fill it in for your product, then apply these self-checks. Each is answerable yes or no:

1. Is the target specific enough that you can name three real companies that fit it — and one that deliberately does not?
2. Does the need describe the customer's pain, not your feature?
3. Would the named alternative's own users recognize your description of it as fair?
4. Would your differentiator survive being read aloud to one of those users?
5. Is your category a thing budgets already have a line for?

If check 1 fails, your first slot says "everyone" in disguise. If check 3 or 4 fails, your differentiator is marketing, not positioning. Rewrite until all five pass; two or three rounds is normal. Ours took more.

Exercise 2 — the non-goals list. Write at least three non-goals for your product. The bar for each entry:

1. Does it name a real product or capability someone will genuinely ask for? ("We won't be slow" is not a non-goal; "no offline mode" is.)
2. Does it state the reason — a mechanism or a cost, not a mood?
3. Would dropping the non-goal change your architecture, your team size, or your market? (If nothing changes, it wasn't worth writing down.)

Keep both artifacts in the repository you intend to build in, not a wiki. They are engineering documents now, and in chapter 0.4 they become the seed of a requirements specification.

Takeaways

If you read nothing else in Part 0, keep these:

• Chat is a systems problem wearing a feature costume. Ordering, idempotency, reconnection, presence, moderation, metering — the estimate misses because the hard parts are invisible in a demo.
• The real cost of building in-house is the second year, paid in on-call and in an engineer who owns messaging forever.
• The market gap is specific: a true chat domain model with a surface small enough to hold in your head — between raw transports and heavyweight platforms.
• Relay's wedge is B2B SaaS with a collaboration surface: five-to-fifty-engineer teams whose users are already authenticated and whose chat is adjacent to their core value.
• Positioning statements and non-goals are engineering artifacts. One compresses every product choice into a testable sentence; the other turns future scope debates into lookups. Both are contracts for the rest of this series.