Skip to content
14 min read

The timestamp that was never on the wire

A phone that loses signal keeps recording. When it reconnects it sends everything at once — and if your API never gave it a field to say when each fix was taken, the server stamps them all with now. The oldest position arrives last and wins.

Vyacheslav Pankratov· Fullstack Developer
Cover art for “The timestamp that was never on the wire”
On this page

TL;DR: Offline GPS tracking is not hard because buffering is hard. If your position API accepts a latitude and a longitude and nothing else, the server has to stamp each fix with its own arrival time — and a phone that buffered an hour of driving through a dead zone replays that hour as a burst of now. The oldest fix arrives last, overwrites the newest, and the map confidently shows a truck somewhere it left forty minutes ago. A payload that cannot carry a capture time makes correct replay impossible rather than merely unimplemented.

We built a freight dispatch platform: drivers' phones report position, dispatchers watch a map, orders move through a status machine. It works. And it has a defect that starts in one line of a request schema and ends with a map that is confidently wrong.

There is no bug in the buffering logic, because there is no buffering logic. That is the point: the defect is latent in the wire contract, so it is inherited by anyone who adds buffering later.

The four hops that turn a track into a lie

The whole chain is short enough to check against your own stack in ten minutes, and it starts with the entire payload — which is the whole problem, in four lines:

PATCH /orders/42/position
{
  "position": { "type": "Point", "coordinates": [4.8952, 52.3702] }
}

One. The position endpoint accepts exactly one field: a point, [longitude, latitude]. That is a valid GeoJSON Point and the shape is fine as far as it goes — but RFC 7946 is a geometry format, not a telemetry format. A Point has no time in it. Nothing in that document was ever supposed to tell you when.

Two. So the server supplies the time itself. When the request lands, the handler writes new Date() into the history row and into the order's current position, both inside one transaction. That transaction is genuinely well built — the live pointer and the historical track can never disagree, which is more than a lot of systems manage. It is simply recording the wrong clock.

Three. The phone, meanwhile, knows exactly when each fix was taken. The Geolocation API hands every position a timestamp alongside its coordinates and accuracy. That value exists on the device, is discarded before the request is built, and has nowhere to go if it were not.

Four. The live position is overwritten unconditionally. There is no comparison against what is already stored — whatever arrived most recently wins, by definition.

Those four are all code facts and all checkable in a few minutes. Here is the consequence, which is where it stops being a curiosity.

Give that client a buffer — which any offline-capable one has, and which is the obvious next feature on a system whose drivers lose signal. It comes out of a tunnel and sends what it held. Fifteen fixes arrive inside two seconds, and the server stamps all fifteen with fifteen nearly identical arrival times. The order of the burst is now the only thing deciding which one the map believes.

If the burst arrives oldest-first, the newest fix wins and everything looks fine. If anything reorders it — a retry, a parallel connection, two requests racing after a reconnect — the oldest fix lands last and wins. The truck jumps backwards, then stays there. And the buffer was supposed to be the fix.

  1. Payload carries a pointand nowhere to put a time
  2. Server stamps arrivalcorrect answer, wrong question
  3. Buffer replays as a burstan hour arrives in two seconds
  4. Live position overwrittenlast to arrive wins, oldest or not
Four hops, and only the last one looks like a bug. The first three are each individually correct, which is why this survives review.

Why does a replayed buffer arrive as a burst of "now"?

Because the only clock in the transaction belongs to the machine that is furthest from the event.

This is the part people find counter-intuitive, so it is worth being precise. There is no bug in the timestamp code. new Date() at the moment of handling is a completely correct answer to the question "when did this request arrive?". The problem is that nobody is asking that question. The dispatcher looking at the map wants to know when the truck was at that point, and the system has quietly substituted a different measurement that agrees with it most of the time — which is the most dangerous kind of wrong, because it is right whenever anyone checks.

It agrees whenever the network is healthy. Arrival time is a fine proxy for capture time when the gap between them is a few hundred milliseconds. The proxy holds right up until the moment it matters, and then it fails in exactly the situation the feature exists for: a vehicle that went out of coverage, which is the case a dispatcher actually needs the history to explain.

The last-arriving old fix wins

The overwrite is unconditional. There is no guard of the form "only update the live position if this fix is newer than the one already stored" — and there could not be one, because the stored fix's recorded time and the incoming fix's recorded time are both arrival times. Comparing them tells you which request the server handled second. That is not the question.

The same missing field breaks deduplication. If a client retries an upload it is not sure landed, the history table gains a second row for a position that happened once. There is no natural key to collapse it against: two rows with the same coordinates and different arrival times are indistinguishable from a vehicle that stopped and idled. So the track silently gains weight, and any figure derived from it — distance covered, time in motion, dwell at a stop — is derived from a corpus that counts some moments twice and places others in the wrong order.

None of this is exotic. It is what happens when a wire contract omits a field that the sender already has and the receiver cannot reconstruct.

How often does the wrong fix actually win?

Everything above is read off the shape of some code, which is a weaker claim than it sounds. A defect you can trace on paper and have never watched fire is a hypothesis. So I built a harness and fired it.

It is a small one, and its limits matter as much as its numbers. An HTTP client and server in a single process over loopback; a buffer of fifteen fixes captured one minute apart; link variance emulated by sleeping inside the handler, drawn from a normal distribution — the shape tc netem applies, so the same experiment can later be re-run over a genuinely impaired link and compared rather than restarted. No packet loss, no retransmission, no congestion control. Five hundred trials per arm, except the one-at-a-time control at sixty — it costs fifteen serialised round trips per trial and its expected result is zero by construction.

The question is not whether the requests arrive out of order. It is whether the live position a dispatcher would see at the end of the flush is the newest fix — because a burst can arrive in any order and still land correctly if the write is guarded. That is the whole argument, so that is what gets measured.

Flush Server Emulated link Wrong position Median staleness
one at a time stamps on arrival 200 ms ± 150 0%
all at once stamps on arrival none 99.0% 7 min
all at once stamps on arrival 50 ms ± 30 99.2% 6 min
all at once stamps on arrival 200 ms ± 150 98.6% 7 min
all at once, 1 connection stamps on arrival 200 ms ± 150 99.6% 5 min
one at a time, 10% retried stamps on arrival 20 ms ± 10 71.2% 5 min
all at once guards on capture time 200 ms ± 150 0%
one at a time, 10% retried guards on capture time 20 ms ± 10 0%

The network is not the mechanism. The concurrent flush loses the newest fix on essentially every trial with no emulated latency at all — on loopback, in one process, on one machine. Adding a bad link changes nothing, because there was nothing to add. "It works on my machine" is not true even on the machine.

Neither is connection parallelism. I added the single-connection arm expecting it to come back clean: HTTP/1.1 keep-alive serialises, so the burst cannot overtake itself on the wire. It came back at 99.6% — no better than six connections on the same link, and the two are close enough that the run cannot separate them. Fifteen requests blocked on one connection are released in an arbitrary order, so the shuffling happens in the client's own dispatch queue before anything is sent. Pinning the pool does not help, and neither will anything else applied downstream of reading the buffer.

The staleness is not marginal. When the wrong fix wins it is the middle of the buffer, not the one just behind the head: the median is five to seven minutes out of a fourteen-minute buffer, and the worst case in every arm was the whole of it. The winner is effectively drawn at random.

Sending one at a time protects — until the first retry. A flush that awaits each request never reorders, which is why a careful client appears to escape. Reject one request in ten and let the client retry it after the rest of the buffer has gone — a retry, not a buffer, and the most ordinary thing a mobile client does — and it is wrong in seven cases out of ten.

The guard costs one comparison and takes it to zero. Same writes, same conditions, one check against the device's own clock. Not "better": zero, by construction, because arrival order stopped being load-bearing.

None of this says anything about a real fleet and should not be read as if it did. It says the defect is not theoretical, that it needs no bad network to fire, and that the field this article is about is what closes it.

  • concurrent flush, one connection99.6%
  • concurrent flush, six connections98.6%
  • sequential flush + 10% retries71.2%
How often the map ends on an older fix than the one it already had, over five hundred trials an arm. The top two are indistinguishable at this sample — pinning the client to one connection protects nothing. The arm not drawn is the capture guard: 0.0%, in every configuration, which is a bar of no length and the whole point.

What a position payload should actually carry

This is the part I would have wanted when I first went looking, so here it is as a minimum rather than as an ideal. Four fields, and each one earns its place by fixing something above.

captured_at — the device's own clock, taken from the platform geolocation API rather than from the moment the request is assembled. This is the field whose absence causes everything in this article.

received_at — the server's arrival time, kept as well, not instead. You need both: the difference between them is the staleness, and it is the only way to tell a live fleet from a replaying one. A system that keeps only capture time trades one blindness for another.

accuracy — the radius the device reports alongside every fix. Without it you cannot distinguish a vehicle that moved 80 metres from one that did not move at all while its position estimate wandered across a parking structure. Drawing both as the same confident dot is a second way to lie with a map.

A client-generated event id — a monotonic sequence or a UUID minted when the fix is captured. This is what makes the write idempotent: a retry carrying an id you have already stored is discarded, and a duplicated upload stops inflating the track.

Field Who sets it What breaks without it
captured_at the device, from the geolocation fix replay is unorderable; the last arrival wins
received_at the server, on arrival staleness cannot be computed at all
accuracy the device, per fix a wandering estimate is drawn as movement
event_id the device, at capture retries inflate the track; no dedup key exists

Which turns the payload above into something a replay cannot corrupt:

{
  "position":    { "type": "Point", "coordinates": [4.8952, 52.3702] },
  "captured_at": "2026-09-05T09:14:22.418Z",
  "accuracy_m":  12.4,
  "event_id":    "b1f2…"
}

The server still stamps its own received_at. It just stops being the only clock in the room.

The device clock is not trustworthy in the way a server clock is — phones drift, users change time zones, some devices lie outright — which is precisely why you keep both stamps and reconcile rather than choosing one. A capture time that disagrees violently with its arrival time is itself a signal worth surfacing, and you cannot surface it if you never wrote it down.

Two clocks, one honest map

Once both timestamps are on the wire, several things that were impossible become ordinary.

The live pointer can be guarded properly: update only when the incoming fix's capture time is newer than the stored one, and a reordered burst stops mattering. The map can show its own uncertainty — "last seen 14 minutes ago" is a different claim from a confident dot, and it is a claim the system can actually support. Replay becomes reconstruction rather than corruption, because the buffer's contents can be placed on the timeline where they belong instead of where they landed. And a duplicate becomes detectable rather than indistinguishable from a stationary vehicle.

None of that requires buffering logic, offline queues or conflict resolution. It requires the sender to say when, and the receiver to write it down.

Adding offline buffering inherits this defect rather than fixing it

The instinct on reading the above is to build a queue on the client. Do that first and you have not fixed anything — you have converted an occasional reordering into a guaranteed one, because every reconnection now replays a batch the server will stamp with a single moment. The wire contract has to carry a capture time before the buffer is worth having.

FAQ

Does this only affect systems with offline support? No — and that is the trap. A system with no offline buffering at all still hits this the moment a client retries, a proxy reorders, or two requests race after a reconnect. Buffering makes the failure larger and more visible; it does not create it. Any system that stamps on arrival has the defect latent, and will meet it the first time the network misbehaves in a way nobody tested.

Can we not just use the arrival time and accept some error? You can, and for a lot of products that is genuinely the right trade. The thing to be clear about is what you are accepting: not a small timing error, but an unbounded one, concentrated exactly in the situations where the history is being consulted. If nobody ever asks "where was the vehicle at 14:20", arrival time is fine. If someone does — a delivery dispute, an incident review, an insurance claim — it is the wrong answer at the worst moment.

How would we know if our system has this? Read the request schema for your position endpoint. If there is no field the device can put its own capture time into, you have it, regardless of what the rest of the stack does. Then look for the guard on the live-position write: if the update is unconditional rather than compared against a stored capture time, a reordered burst will move your map backwards.

Is this expensive to fix? The wire contract change is small — a nullable column, a schema field, a client that sends what it already has. What costs is the reconciliation you now have to design: which clock wins when they disagree, what tolerance counts as drift rather than staleness, and what the map shows while it is unsure. That is a day of decisions rather than a week of code, and it is much cheaper before the history table is full of rows that cannot be reinterpreted.

The three things worth taking away

  1. A payload that cannot carry a capture time makes correct replay impossible, not merely unimplemented. Adding buffering later inherits the defect rather than fixing it.
  2. Keep both clocks. Capture time answers "when was the truck there"; arrival time answers "when did we hear about it". The difference between them is the staleness, and you cannot compute it if you only stored one.
  3. Guard the live write on capture time. An unconditional overwrite means a reordered burst moves your map backwards, and no amount of client-side care prevents it from the server's side.

The next piece in this series is about what happens to that staleness once it does arrive — the freshness field the dashboard never read, which is the same failure one layer up.

The rest of what this breaks — the dashboard that cannot tell, the table that keeps every version of it — is on freight telemetry that survives a bad link.

If you are looking at a tracking system and want a second pair of eyes on where it loses the truth, that is what we do.

Was this helpful?

// Build it

Running this in production?

Telemetry fails quietly — a timestamp that never reached the wire, a track you cannot draw, a fleet that reconnects all at once. Tell us what your map is showing and when; we reply within a day with a concrete next step.