Your reconnect hint sets your data loss, or it sets nothing at all
A server-sent events stream that omits the id field loses everything published while it is away, and the amount is arithmetic rather than luck. The reconnect hint you tuned decides that number — or decides nothing whatsoever, depending on one line you may not have written.
On this page
TL;DR: A server-sent events stream that omits the
id:field loses everything published while it is away, and the amount is arithmetic rather than luck: lost events per reconnect is the gap divided by your publish interval. We measured 100.6 against a predicted 100. Send the id, keep a buffer that still reaches back that far, and the same severed connection costs zero. So the reconnect hint you tuned is a loss dial in the first case and a no-op in the second — and we expected the opposite going in.
Every long-lived stream gets cut. A proxy times out an idle connection, a load balancer drains a node, a laptop changes networks, a deploy rolls a pod. None of that is exceptional and none of it is worth engineering away, because the reconnect is automatic and takes a moment.
What is worth knowing is the bill. Between the sever and the next successful read, your publisher keeps publishing into a socket nobody is holding, and whether those events are gone or merely delayed is decided by a field you either wrote or did not. The mechanism is in the specification and in every tutorial. The size of the bill is in neither.
So we measured it, on a harness that does nothing but sever a stream on a schedule and count what came back. Two of the three findings were the ones we went looking for. The third contradicted the claim we had written down before the first run.
What one reconnect actually costs
The arithmetic turns out to be the whole story, and it is boring enough that nobody seems to have bothered publishing it: while you are disconnected, you miss whatever was published. Divide the gap by the publish interval and you have the number.
Our publisher emits every 20 ms. The client obeys the reconnection time the server advertises, so the
gap is that hint plus the cost of getting back — negligible on loopback, which is the point at which
these numbers become a floor rather than an estimate. Three hints, no id: field, medians of three
30-second runs:
| Reconnect hint | Predicted loss per reconnect | Measured |
|---|---|---|
| 100 ms | 5 | 5.7 |
| 500 ms | 25 | 25.6 |
| 2 s | 100 | 100.6 |
That is the mechanism reproducing itself three times. There is nothing probabilistic about it: the stream is not dropping events, it is publishing them to nobody, and "how many" is a question about your own cadence rather than about the network.
Which also means the honest unit is per reconnect, not per hour and not as a percentage. We could tell you our harness lost 46.9% of everything published, and it did — but only because it severs the connection every two seconds, which is a number we chose. Change how long we watch and that percentage moves several points while the per-reconnect figure holds. A percentage here is a fact about the experiment. The per-reconnect figure is a fact about SSE.
Why does the reconnect hint change the loss at all?
Because the hint is the gap. A server can advertise a reconnection time on the stream itself, and the client waits that long before coming back:
retry: 2000
data: {"symbol":"ABC","price":41.02}
data: {"symbol":"ABC","price":41.06}Two seconds of silence at 20 ms per event is 100 events, and that is what the table shows. Reconnect sooner and you are away for less time, so you miss less. Stated plainly it sounds like something nobody needed to measure.
We had written down the opposite, and it is worth being specific about that. The claim we intended to publish was an inversion: that a shorter hint would lose more, because it buys more frequent unrecoverable gaps — each one small, but many of them. It is not a stupid argument. It is just wrong, and it is wrong in a way that would have survived a plausibility check and gone out with a number attached.
What saved it was procedural rather than clever: the claim went into the harness's own header as a hypothesis before the first run, so the output could not be quietly reshaped to fit it. The run refuted it by 41.6 points. Reconnecting more often does not accumulate loss, because the loss was never per-reconnect-event — it is per-second-away, and returning sooner means fewer seconds away.
- Event carries an idthe server's half of the contract
- Client stores itas the last event ID
- Connection severedproxy, deploy, network change
- Reconnect with Last-Event-IDthe client's half
- Buffer still reaches itthe gap is replayed — zero lost
- It has been droppedfresh events only, no error
The same dial does nothing once you send ids
Now add one field per event and honour the header the client sends back. The spec's mechanism is
plain: a server may attach an id: to each event, the browser remembers the last one it saw, and on
reconnection it sends that value in a
Last-Event-ID header.
A server that keeps a short buffer can then send the gap.
Same harness, same severs, same three hints:
| Reconnect hint | Lost per reconnect, no ids | Lost per reconnect, ids + replay |
|---|---|---|
| 100 ms | 5.7 | 0 |
| 500 ms | 25.6 | 0 |
| 2 s | 100.6 | 0 |
Zero, not "almost zero" — every event published during every gap arrived, on all three arms, across 1,500 events per arm. Our first run showed one lost event and we chased it: it was the harness stopping its publisher after its observer, so the feed ticked once into a window nobody was watching. That defect is fixed and worth naming, because a floor made of your own shutdown reads exactly like the remedy's residual cost.
The interaction is the part worth carrying away. The reconnect hint is a real lever with a measurable effect on data loss — in the left column. In the right column it is decorative. So the question "what should our retry be?" has no answer until someone says which column the system is in, and that is a different question from the one usually being asked in the thread.
What this run does not license
The harness severs the connection on a fixed schedule, so it measures what a reconnect costs and never how often reconnects happen to you. Nothing here says a thing about your outage rate.
It is one process on loopback with no proxy, no TLS and no real network, so a reconnect costs a local accept and a handler setup. On a real path it also carries DNS, TCP, TLS and whatever an intermediary adds — every gap is longer than ours and every loss figure is therefore a floor.
The one thing left is what happens past the buffer, and we have that number now — it was this article's own stated gap, and closing it took one flag.
Same arms, ids on, a two-second gap costing about a hundred events, with the server's replay buffer swept from comfortably covering that gap to not covering it at all:
| Buffer holds | Covers | Lost per reconnect |
|---|---|---|
| 512 events | 10.2 s | 0 |
| 200 events | 4 s | 0 |
| 100 events | 2 s — exactly the gap | 57.7 |
| 50 events | 1 s | 100.6 |
| 10 events | 0.2 s | 100.4 |
In our harness, replay is all or nothing. A reconnect the buffer still reaches costs zero; one it does not costs the entire gap — precisely as if no ids had been sent, because our server resumes only from an id it still holds and otherwise sends nothing from the buffer. Under that policy there is no partial replay, no best effort, no degradation curve; a server that sends whatever it still holds would lose only the part of the gap that fell out. Divide the loss by the number of reconnects that fell outside the window and you get 100.6 events against a gap of 100: each one paid in full.
Which makes the middle row the interesting one, and a trap. At a buffer sized exactly to the gap the average reads 57.7, which looks like graceful degradation — and there is none. Some reconnects fell inside the window and some outside; every one of them got either everything or nothing.
It is also the only number here that does not reproduce. A repeat of the same run put that row at
43.3 rather than the 57.7 the committed results.txt carries, because it is decided by how many reconnects happened to land on each side of a
boundary — while every other row in the table moved by less than a tenth of an event. So the middle
row is not a measurement of anything: an averaged metric at that boundary describes a state the
system never occupies, and if it is the metric on your dashboard it will describe it to you every
time you look, at a different value each time.
What to do with this instead of tuning the hint
Establish which regime you are in first, because it decides whether the rest of the conversation matters. Three questions, in order:
- Does the stream carry an
id:on every event? Not on some. A gap that spans an un-numbered event leaves the client's resume point behind it. - Does the server retain anything to replay from, and how far back does it reach? The spec places no obligation here at all — replay is a thing you build, not a thing you get.
- Does the buffer's reach exceed your realistic worst gap, including the slow reconnect, not the median one? This is the one that turns a working system into a silently lossy one as traffic grows, because the buffer is sized in events and the gap is measured in seconds.
If the answers are yes, yes and yes, the reconnect hint is a latency knob and you can set it by how quickly you want the screen to come back. If any answer is no, the hint is your loss rate, and the arithmetic above tells you what you are choosing.
In our harness, replay is all or nothing
FAQ
Does a shorter retry hint lose less data? Yes, when the stream sends no ids — and this is the finding that contradicted our own prediction. The loss is set by how long you are away, and a shorter hint means less time away. Once ids and a sufficient buffer are in place, the hint changes nothing about loss; it only changes how quickly the stream resumes.
Do we get replay automatically by adding the id field?
No. The id field and the Last-Event-ID header are the protocol's half. The server side — retaining
recent events and sending the ones a returning client missed — is application code you write, and the
specification
requires none of it.
What happens when the gap outlives the buffer? Replay degrades to a gap, quietly: in our harness the client asks to resume from an id the server has dropped and gets fresh events instead. There is no error, and the client cannot tell the difference from a normal resume. The buffer sweep above measured what that costs under our server's policy — resume only from an id it still holds, otherwise nothing from the buffer: the whole gap, as much as a reconnect with no ids at all. A server that sends what it still holds loses only the older part; either way nothing reports it.
Is this specific to SSE? The arithmetic is not. Any fan-out where the publisher keeps publishing during a client's absence has the same bill, and the same two regimes depending on whether the transport can replay. What is specific to SSE is that the resume mechanism is standardised and free, and still has to be turned on at both ends.
The three things worth taking away
Loss during a reconnect is arithmetic, not luck. Gap duration over publish interval, reproduced to within a percent at three different hints. If you know your cadence, you can compute your exposure today without running anything.
The reconnect hint is either your loss dial or a no-op, and which one depends entirely on whether the stream carries ids and the server can replay from them. Tuning it without knowing that is tuning a control that may not be connected to anything.
A remedy that costs nothing to state is still not free. The id field is one line and the header is handled for you; the buffer, its size, and what happens when a gap outruns it are yours to build and yours to get wrong. Past the window you lose at least what fell out of the buffer — and, in our harness, all of the gap — and nothing reports it, so the buffer's reach, not the hint, is the number worth watching.
For a price stream this is one half of the picture on market feeds that reconnect without their symbols; the other half is the subscription that never comes back.
If you cannot say today what one reconnect costs your stream in events, that is a cheap thing to have looked at.
