Video not publishing, audio fine — the bug that only appears at production framerate

What to measure, and what the number should be#

This is a two-minute audit, and it generalises well beyond WHIP:

1. Find every buffer-count queue on a path that must survive a variable-length stall. Grep for max-size-buffers. Negotiation paths, reconnects, and anything feeding a network sink all qualify.
2. Divide by the framerate that path actually runs at in production — not in dev. That is your real depth.
3. Compare it to your worst-case consumer-start delay. For WHIP/ICE/DTLS that is seconds. If the depth is milliseconds, you have this bug; it is simply waiting for a slow handshake to expose it.

The fix is to stop expressing the buffer in frames:

Measured on 2026-07-28 on an otherwise identical 1080p60 profile — same encoder, same bitrate, same network path, only the queue changed — the buffer-count version produced a handful of packets and then a sender timeout, while the 2-second time-based version sustained the full target bitrate with no timeout. One variable, so the result attributes cleanly; a number without that sentence attached is a number the next person is entitled to distrust.

Three things worth knowing before you copy that block:

Keep a byte ceiling. Do not set all three limits to zero. This is the part that is easy to get wrong, because "size it in time" sounds like it should replace the other two. A time-based queue computes fullness from buffer timestamps — so if a buffer ever arrives without a valid PTS, or with a non-monotonic one, the time limit cannot be evaluated and the leak never fires. With max-size-buffers=0 and max-size-bytes=0 you have then removed every bound at once, and a stalled consumer grows the queue until something else breaks. Set the byte ceiling far above two seconds of any bitrate you actually run, so it never leaks in normal operation and exists purely as the backstop for the pathological-timestamp case.

Two seconds was the validated threshold, not a safety margin. It passed with no headroom above it. If timeouts reappear under harsher conditions than your test exercised — relayed connections, real packet loss — that number is the first place to look, not the last.

Do not paste it onto a raw-video queue. Downstream of the encoder you are buffering compressed frames, and two seconds is cheap. Upstream of the encoder you are buffering uncompressed ones, and two seconds of raw 1080p is an enormous amount of memory. A queue in that position still wants a time-based bound — just a much smaller one, chosen for its own position in the graph.

Is it actually this, and not something else?#

Video dead, audio alive has several causes, and a buffer is only one of them. Before you change a queue, spend two minutes ruling out the others — they present almost identically and the fixes have nothing in common. Four candidates, and the one signal that separates each from the rest:

Each in more detail, because the tells are what you actually act on.

A profile the receiver won't decode. If the encoder emits H.264 High profile, B-frames, or a long GOP and the receiver is strict about it, you get a live track that never paints. Safari is the usual suspect. The tell: it fails consistently on one browser and works everywhere else, whereas the buffer bug tracks network conditions rather than client. Check the negotiated profile in the SDP answer.

No keyframe ever arrives. Some receivers only request a keyframe once and give up. If the encoder's GOP is long and the first request is missed, the track stays black indefinitely with healthy stats climbing. The tell: bytes received keeps rising and frames decoded stays zero — nothing was dropped, it just isn't decodable. The buffer bug looks different: bytes received stalls near zero.

Media never actually flowed. A DTLS handshake that completes for audio's m-line and fails for video's, an SRTP mismatch, or a firewall that permits one port and not another, all produce a video track that exists in the session and carries nothing. The tell: candidate pairs and DTLS state differ between the two m-lines.

Only then, the buffer. Its signature is specific and worth memorising: the symptom correlates with how long negotiation took, not with the client, the profile, or the content. Fast local connection, fine. Relayed or cold connection, dead. That correlation is the thing no other cause on this list produces.

How do you confirm it rather than guess?#

Three instruments, in the order that costs least.

The receiver's own stats. In Chromium, chrome://webrtc-internals gives you a section per peer connection (Chrome documents the page and its graphs). Watch iceConnectionState progress through checking to connected — if it stalls in checking and then goes failed, you have a connectivity problem, not a buffer one. If iceGatheringState never reaches complete, your STUN/TURN configuration is wrong. Both of those exonerate the queue.

Bytes versus frames. The distinction that separates the three causes above: compare bytesReceived against framesDecoded on the inbound video track — both are standard fields on RTCInboundRtpStreamStats (W3C WebRTC Statistics), so this works from any client, not just Chromium's debug page. Climbing bytes with zero decoded frames means the data arrived and could not be decoded — a profile or keyframe problem. Bytes flat near zero while the connection reports connected means the data never left, which is where a starved queue lands you.

The pipeline's own view. On the sending side, GST_DEBUG=queue:5 reports each queue's fullness and every leak it performs; a queue that is dropping during negotiation says so explicitly. The GStreamer debugging documentation covers the level syntax. Two things worth logging alongside it: how long the sink took to reach PLAYING, and how long ICE took. If the gap between them exceeds your queue depth, you have your answer without changing anything.

Then prove it by changing one thing. Keep the encoder profile, bitrate and network path identical, switch only the queue from buffer-count to time-based, and compare packets sent and whether the sender times out. One variable, unambiguous result — and it takes minutes, which is why it beats reasoning about it for an afternoon.

Is the leaky queue the real culprit?#

No, and it is worth being precise, because "remove leaky" is the tempting wrong fix. A leaky queue is correct on an output path: it means a stalled consumer cannot apply backpressure all the way up your pipeline and wedge the encoder or the capture device. Take the leak away and you trade a dead video track for a stalled pipeline, which is worse.

The leak is what makes the shallow depth fatal rather than merely slow. Fix the depth and keep the leak.

Is this a GStreamer problem?#

No — GStreamer just names the knob clearly enough that you can see the hazard. Any pipeline that bounds a queue by item count ahead of a consumer that starts late has the same failure, and the tell is always the same: the bound's unit is not time, so its real depth is a function of a rate somebody else controls.

FFmpeg has both shapes side by side, which makes the point better than an argument does. -thread_queue_size bounds an input queue in packets — a count, exactly like max-size-buffers, with a small default and a real depth that shrinks as packet rate rises. -rtbufsize bounds a real-time input buffer in bytes. Neither is expressed in seconds, so both need the same arithmetic before you can say how long a stall they survive: divide by the rate that path actually runs at in production.

The general form is worth carrying beyond media entirely. A bounded buffer sized in items, in front of anything that may start consuming late — a connection handshake, a lazily-spawned worker, a consumer that waits for a lease — is a latent, rate-dependent bug. It behaves on the machine with the slower rate and fails on the faster one, which is the wrong way round from every intuition about performance problems.

How do you stop it coming back?#

Fixing the queue you found is the small half. This class of bug is worth a rule, because the next one will be introduced by someone who never read this article — and it will not look like a bug in review. A buffer count is a plausible, tidy-looking number; nothing about max-size-buffers=3 announces that its meaning depends on a variable set in a different file.

Make the unit a review rule, not a preference. Any queue on a path that must survive a wait of unknown length is sized in time. Negotiation, reconnection, renegotiation, relay allocation, a downstream consumer that starts lazily — all of these qualify. Buffer counts stay legitimate where the consumer is always ready and the framerate is fixed and known, which is most of a pipeline; the rule is narrow on purpose, because a rule that bans a whole primitive gets ignored.

Grep for it in CI, not in code review. One line — grep -rn "max-size-buffers" --include=*.py --include=*.c — turns "somebody has to remember" into a list that a human decides on. Reviewers miss this reliably, because the number looks fine and the dependency is invisible at the diff. A check that merely lists the occurrences is enough: the judgement of whether a given queue sits on a stall-prone path is not automatable, but noticing that a new one appeared is.

Test at the production profile, or accept that you are not testing this. The reason this survived review and staging is that both ran a lower framerate, where the same code had twice the headroom and a faster handshake. Any acceptance test for a publish path should run at the profile production actually uses — resolution and framerate — and ideally through a relay, because that is where the negotiation window is long enough to matter. A green test at 720p30 on localhost tells you almost nothing about 1080p60 through TURN.

Record the number and where it came from. Two seconds was measured, not chosen, and a number whose provenance is lost gets "tidied" by the next person who finds it arbitrary. A one-line comment naming what was tested — profile, network path, result — is what stops a future cleanup from silently reintroducing the bug. The same applies to the byte ceiling: without the note explaining it exists for the missing-PTS case, it reads like a leftover.

What if you can't change the queue?#

Sometimes the pipeline is not yours: a vendor SDK, a managed sink, a binary you can configure but not recompile. The mechanism still applies, so the levers just move.

Shorten the wait instead of deepening the buffer. The failure condition is queue_depth < negotiation_window, and the right-hand side is often more compressible than people assume. Pre-gathering ICE candidates, keeping a warm connection, or removing an unnecessary relay hop all shrink the window. If the handshake completes in 200 ms, a 50 ms buffer stops mattering nearly as much.

Start the media late. If you can delay the encoder until the sink is actually ready to pull — holding the pipeline in PAUSED, or gating the source — then there is no backlog to discard, because nothing was produced during the wait. This is the cleanest workaround when it is available, and it is often a one-line state change.

Raise the keyframe rate for the first seconds. Since what kills the track is losing the keyframe, a shorter GOP during connection setup means the receiver gets a decodable frame sooner after it starts pulling. It costs bitrate, so revert it once the connection is established rather than living with it.

Failing all three, make the failure loud. If you cannot fix it, at least stop it presenting as a mystery: log the gap between "sink reached PLAYING" and "first packet pulled", and alert when it exceeds the queue's depth. That turns a silent black video track into a named condition with a number attached — which is the difference between a support ticket that gets solved and one that gets closed as "could not reproduce".

If the honest answer is that the pipeline needs owning rather than patching, that is a scoping question before it is an engineering one — our estimator puts a range on it in a couple of minutes, starting from real-time video as the domain.