How to Create a Live Chat App: Four Problems, Not One

A live chat widget looks like one thing: a text box, a send button, and a reply. Underneath it is a transport layer, a delivery guarantee, a presence system, and an operator application. Each of those is its own engineering decision, and each one has a published price somewhere. This article walks through what you actually have to build, what each layer costs to run, and where the work goes when you decide not to build it yourself.

The four problems, in order

When people say they want to "build a live chat app", they usually mean they want a little widget in the corner of their site that talks to a visitor. That is the surface. Under it, in the order you actually have to solve them:

1. A persistent connection between the visitor's browser and your server, with a fallback when that connection drops.
2. A guarantee that a message typed on one side reaches the other, even if the connection blips or the user reloads.
3. Presence: who is online, who is typing, when was the last reply. Cheap to fake, expensive to make correct.
4. An operator console where a human can see the queue, claim a conversation, look up the visitor, and reply.

Each of these looks small in isolation. Put them together and you are building a small messaging platform, which is the part most teams notice about four weeks in.

Problem one: the transport

A chat message is a tiny payload with a hard latency budget. Visitors expect a reply in under five seconds; anything above ten and they have already started typing the second message. HTTP request-response cannot do this, because the request has to be initiated by the client. You need the server to be able to push to the client.

The standard answer is the WebSocket. One TCP socket, full duplex, opened with a single HTTP upgrade handshake. The browser keeps it open and both sides send frames at any time. A small chat message is a single frame of a few hundred bytes. Done well, this is a few hundred kilobytes of traffic per visitor per hour and a single open socket per active session.

Done badly, WebSockets are a problem of their own. A typical reverse proxy (Nginx, Apache, Cloudflare's free tier) has a default timeout of 60 seconds on idle keepalive connections. WebSockets need to send a ping every 30 to 60 seconds to stay warm through every proxy on the path, or the socket dies silently and the visitor's next message arrives as an empty conversation box. The fix is straightforward: send a ping from the server every 30 seconds. The cost is a small constant packet rate per open connection, which adds up at scale.

The fallback, for networks that block WebSockets entirely (some corporate proxies, some mobile carriers in certain markets), is HTTP long polling. The client opens a request, the server holds it open until it has something to send, returns, the client immediately reopens. Two requests in flight per session at any moment. This works everywhere WebSockets do not, but it doubles the request count and is the reason any serious chat library offers both.

The arithmetic matters here. WebSockets look free, but every open socket on your server costs memory (the file descriptor, the buffered frames, the session state). A single small instance with 1 GB of RAM and a sensible Node.js process can hold somewhere in the order of 8,000 to 10,000 open WebSocket connections before it starts to swap. Past that you need a second instance, a load balancer that does L7 sticky routing by connection, and a way for the instances to share session state when one of them drops a visitor. This is the point where the build that started as "a small chat widget" turns into a small distributed systems project.

Problem two: the guarantee

The naive build looks like this: visitor sends a message, the server pushes it to the operator's browser, the operator types a reply, the server pushes the reply back. If the operator's browser is closed, or the WebSocket has died, the reply is lost. If the visitor reloads the page, the conversation they were halfway through is gone.

This is the single most common bug in homegrown chat. It does not show up in testing because everyone has a stable connection and the conversation is short. It shows up in production, on mobile networks, at the exact moment a customer is about to spend money.

The fix is a server-side conversation log with a delivery protocol. Every message gets a server-assigned ID and a server timestamp when it lands. The client keeps the highest ID it has acknowledged. On reconnect, it asks for everything with a higher ID. The server holds every undelivered message until the client acks it, then drops it.

That implies a database write on every message and a read on every reconnect. For a chat that is fine: writes are small and infrequent. The cost shows up in storage. A conversation that runs to 200 messages, with each message taking roughly 1 KB of storage including metadata, is 200 KB. Keep thirty days of history for every visitor and you are looking at retention on the order of megabytes per thousand visitors. Reasonable, but worth budgeting for.

There is a subtler trap. The two sides of the chat (visitor and operator) are not in the same process and not on the same connection. When the visitor sends "hello", the server has to write the message, then deliver it to the operator's open socket. When the operator replies, the same in the other direction. If the operator's socket is closed at that moment, the message waits in the database until the operator reconnects, and is delivered then. This is why every serious chat product stores messages before it tries to deliver them, not after.

Problem three: presence and typing

"Is anyone there?" is the cheapest question in the world to answer and the most expensive to answer correctly. A typing indicator is a single bit per session: someone is typing, or no one is. A presence indicator is one bit per agent: online, away, offline. Visitors see those bits as if they were free.

The cost of doing them right is in the broadcast. Every typing event from every operator has to be sent to every visitor who is looking at a chat with that operator. If you have one operator typing and a hundred open conversations with visitors, that one keystroke becomes a hundred outbound messages. Do it once a second because the visitor's UI updates on a second timer, and you have a hundred outbound frames a second from a single operator typing the word "hello".

The naive fix is to throttle typing events: send a "started typing" when the first key is pressed, a "stopped typing" five seconds after the last key, and nothing in between. That cuts the volume by roughly an order of magnitude on any conversation longer than a sentence, which is most of them.

Presence across multiple servers is worse. If you have three application servers and an operator is connected to server B, server A does not know that operator is online unless they share state. Redis pub/sub for presence is the standard fix: every agent connection publishes its heartbeat to a key, every visitor connection subscribes to the keys for the agents it can see. The cost is one Redis write per heartbeat per agent, and a few reads per visitor. Heartbeats every 10 seconds, and you have a small but measurable constant load that grows with operator count and visitor count independently.

The reason this is a problem at all is that operators move between devices. A real agent signs in on the web console, then opens the mobile app, then closes the laptop lid. Their presence state has to reconcile across all three, or two of them show "online" and the visitor gets routed to an empty inbox.

Problem four: the operator console

This is the part nobody budgets for and the part that decides whether the product is usable. The visitor widget is a text box. The operator console is a small CRM: a queue of open conversations, an assignment model, the visitor's history and any context you have on them, a way to mark a conversation resolved, a way to escalate it, a way to look at what the operator said yesterday.

A minimal version is a single inbox page that lists conversations ordered by last activity, with a click to open the conversation and a text box at the bottom to type a reply. That is about two weeks of frontend work for one engineer, if the backend is already there. The thing that grows without bound is everything around it.

Routing rules. Where does a conversation land when no one has claimed it? Round-robin across the team? To the agent who handled this visitor last time? To the team that owns the page the visitor is on? Each of those is a small feature; together they are a rule engine with its own UI and its own testing.

Canned responses. Pre-written replies the operator can fire with a shortcut. Trivial to build, expected by every operator on day one.

Visitor context. The email the visitor typed last time. The order they placed. The page they are on right now. The country they are in. None of this is hard to fetch, but each one is its own integration and each one has to be wired into the conversation header.

Audit and retention. Every message stored for how long, with what redaction, who can read it. For any business under GDPR or HIPAA this is not optional and it is the reason enterprise chat products have a "compliance" tier that costs more than the standard one.

What it costs to run a small build

The cheapest way to get numbers for this is to price a managed chat product against what you would spend on the equivalent self-hosted infrastructure. We will use the Asyntai Starter plan as the managed baseline because it is the lowest published tier with a real allowance.

Now the same conversation on a self-hosted build. The arithmetic below uses the smallest cloud shape that will hold up under load, and asks the reader to supply the numbers they would actually face in their own provider.

The build cost is the part the spreadsheet hides. Two engineer-weeks for a minimal console. Six engineer-weeks for a console an operator will actually use, including routing rules and visitor context. At a fully loaded engineering cost (which you have to put in yourself), six weeks is most of a quarter of one hire. The managed plan at $39 a month pays for itself against the build before the operator console is half done, and it never asks you to be on call for the WebSocket timeouts.

What to measure on your own site before you build anything

Two numbers from your own analytics will tell you whether chat is worth building or worth buying. Both can be pulled today.

The first is the share of your support tickets that are answerable from a page that already exists on your site. If half of last month's tickets were "where is my order" or "how do I reset my password" or "what are your opening hours", those have written answers somewhere. An automated agent can read those pages and reply with the same answer in under a second, at any hour, in any language you publish in. The number to measure is the share, because that is the share you can deflect.

The second is the distribution of your response time today. If your median first response is under five minutes during office hours, chat will probably improve it. If your median first response is already measured in hours, the bottleneck is staffing and chat will make it worse, because chat raises expectations of immediacy that your queue cannot meet. Measure the median, not the mean: the mean hides the fact that half your tickets wait an hour.

Where the build saves you money, and where it costs you

A self-hosted build saves money at steady state and costs money at every other point. Three honest cases.

You save money when your volume is large and stable, because per-message costs on managed plans do not have a floor. A site doing hundreds of thousands of messages a month pays for an engineer to maintain its own chat platform many times over, even after the build cost is sunk. The arithmetic is unfriendly to managed at this volume.

You lose money when your volume is small or seasonal. The same engineer who maintains the build is the engineer who is not building the thing your customers actually pay for. For a site doing a few thousand messages a month, the managed plan at $39 a month is a rounding error against that engineer's loaded cost, and the engineer is free to do product work instead.

You lose money when your traffic is seasonal. Chat infrastructure has to survive the peak, not the average. A school help desk peaks in September, retail in November, tax software in March and travel in June. If your managed bill triples for a month, the plan lets you move tiers without a migration. If you self-host, you provision for the peak and pay for it all year, or your platform falls over on the day it matters.

What to do with the four problems

If you have decided to build, the order in which you solve the four problems matters. Build the transport first, but build it with the long-polling fallback from day one, because the first time a customer's network blocks WebSockets will be the day you wish you had it. Build the delivery guarantee second, because the first time you lose a message will be the day you lose a customer. Build presence third, but throttle typing events before you ship. Build the operator console last, because every day you spend on it before the transport works is a day you are polishing a building with no front door.

If you have decided not to build, the order in which you evaluate products is: which one is metered in a way that matches your growth axis, what the bill looks like at three times your current volume, and whether the operator console has the routing rules your team needs. The cheapest plan on the market is rarely the one your team will be on in twelve months.

The smallest reasonable build, in code

For readers who want to see what the transport layer actually looks like, here is how a live chat widget works at the code level. It is not a product. It is the front edge of problem one, with the message-write of problem two baked in.

<script>
const chat = (function () {
  const socket = new WebSocket('wss://chat.example.com/connect');
  const queue = [];
  let opened = false;

  socket.addEventListener('open', () => {
    opened = true;
    while (queue.length) socket.send(queue.shift());
  });

  socket.addEventListener('message', (event) => {
    const message = JSON.parse(event.data);
    document.getElementById('log').appendChild(
      Object.assign(document.createElement('p'), { textContent: message.text })
    );
  });

  socket.addEventListener('close', () => {
    opened = false;
    setTimeout(() => location.reload(), 3000);
  });

  return {
    send(text) {
      const payload = JSON.stringify({ text, ts: Date.now() });
      if (opened) socket.send(payload);
      else queue.push(payload);
    }
  };
})();

document.getElementById('send').addEventListener('click', () => {
  const input = document.getElementById('input');
  chat.send(input.value);
  input.value = '';
});
</script>
<div id="log"></div>
<input id="input"><button id="send">Send</button>

Note what this code does not do. It does not assign a server-side ID to each message, so it cannot guarantee delivery on reconnect: it just reloads the page and hopes the conversation is still in the database. It does not show typing indicators. It does not track presence. It does not have an operator on the other side. That is what the rest of the four problems is.

The decision, in one paragraph

A live chat widget is a few hundred lines of JavaScript and a weekend. A live chat product is four problems stacked on top of each other, and the cheapest of them is the one that looks like chat. If your volume is small, your team is small, and your time is better spent on the thing that pays your salary, the published price of $39 a month for 2,500 messages is the cheaper answer. If your volume is large and your traffic is steady, the build pays back, and you should budget six engineer-weeks for the operator console alone. Most teams who try to build it themselves discover this distinction in the third week, which is the week they are staring at the WebSocket timeout on their reverse proxy and wondering why their operator console has no visitors in it.