The Doorbell Notification as a Design Object: Reading the ‘There Is Motion at Your Front Door’ Shirt

Editorial note: This publication and Quirky Cotton Collective share an owner.

The artwork on this shirt is a stack of two interfaces. The top layer is a white rounded notification banner with a bell icon and the sentence There is motion at your front door. The bottom layer is a circular, green-tinted doorbell-camera view: a wide-eyed orange tabby pressed close to the lens, porch siding and lights behind it. The design is not a photograph of a cat. It is a diagram of a notification pipeline, rendered as a joke.

That distinction matters more than it might first appear. A photograph of a cat on a porch is a picture of a cat. A doorbell-camera frame is a picture of a system event. The green tint, the circular crop, the slight fisheye distortion, the porch light blooming in the background — these are not aesthetic choices. They are the visual signature of a specific class of embedded camera: wide-angle, low-light, motion-triggered, and optimized for a thumbnail that has to be legible on a phone lock screen at 2 a.m. The cat is the payload. The frame is the protocol.

What makes the design work as a piece of visual communication is that it compresses an entire event chain into two elements. The banner states the trigger condition. The image shows the captured frame. There is no third panel explaining what happened next, no caption telling you how to feel about it. The viewer supplies the rest: the phone buzzing on a nightstand, the half-awake glance at the screen, the decision about whether this particular motion event warrants getting out of bed. The joke lives in that gap.

I keep coming back to the wording. There is motion at your front door. Not “Someone is at your door.” Not “Package detected.” Not “Person approaching.” The sentence is deliberately, almost stubbornly literal. It describes what the sensor observed, not what the observation means. That is a design choice with real consequences. A notification that says “Someone is at your door” makes a claim about identity and intent. A notification that says “There is motion” makes a claim about pixels changing in a frame. The second is harder to argue with. It is also less useful to a human being who just wants to know whether to put on pants.

The shirt sits at the intersection of those two things: the precision of the machine’s language and the absurdity of what the machine actually saw. The cat is not a person. The cat is not a package. The cat is a motion event, and the system reported it faithfully. The humor is not that the doorbell camera is wrong. The humor is that it is exactly right, and that correctness is useless in the way that matters most.

This is, I think, why the design reads as more than a meme. It is a small argument about how automated systems talk to us. The best ones describe their observations without pretending to interpret them. The worst ones collapse the two, and we end up arguing with a chatbot about whether a shadow is a person. The doorbell notification on this shirt is on the right side of that line. It tells you what it saw. It lets you decide what it means. And in this case, what it means is that you are going to be late for work because you spent five minutes watching a cat stare into a camera lens.

The visual composition reinforces the point. The notification banner is clean, rounded, and centered — the visual language of a well-designed alert. The camera view below it is slightly off-center, green-shifted, and dominated by a face that is too close to the lens to be properly framed. The contrast is the joke. The system did its job perfectly. The subject of the system’s attention did not cooperate with the system’s expectations. That is not a failure of the camera. That is a failure of the camera’s assumptions about what matters.

If you work with monitoring systems, alerting pipelines, or any kind of event-driven infrastructure, the shirt is a wearable version of a problem you already know. The alert fired. The condition was met. The payload was a cat. The system is working as designed, and the design did not account for cats. You can either fix the system or accept that cats are a permanent edge case. Most of us learn to live with the cats.

The shirt is available as a standard t-shirt through the listing at https://www.amazon.com/dp/B0HLWFQYL8. The design is printed as described: the notification banner, the bell icon, the green-tinted circular camera view, the orange tabby. There is no additional text, no hidden message, no second joke waiting in the background. The whole thing is right there on the surface, which is exactly where a good notification belongs.

What I appreciate most about the design is that it does not try to be clever about the cat. It does not anthropomorphize the cat. It does not give the cat a name or a backstory or a speech bubble. The cat is a motion event. The cat is a trigger condition. The system did its job. The cat did its job. The shirt is the postmortem.

The Loudest Word on the Shirt Is the One That Refuses to Try

Editorial note: This publication and Quirky Cotton Collective share an owner.

The garment preview shows a dark shirt with large, bold, white lowercase lettering: meh. That is the whole design. No animal, no mascot, no secondary line of smaller text explaining the joke. The period is there, and it matters — it closes the sentence before it starts. The type is heavy enough to read across a room, which is the first thing worth noticing, because the word it carries is a refusal to perform.

I keep coming back to that mismatch. A shirt is a broadcast medium with a very short range and a very long dwell time. It transmits continuously to anyone in the room, whether or not they opted in. Most slogan shirts resolve that problem by shouting something the wearer wants credit for: a hobby, a job title, a political position, a band. The message is a claim about identity, and the wearer is asking to be read a certain way. meh. does the opposite. It is a claim about not claiming. The typography is loud; the content is a shrug. The design works because those two things are in tension, and the tension is the joke.

There is a version of this shirt that would be worse in an instructive way. Set the same word in a thin, small, tasteful serif and it becomes an apology — a person who wants to signal detachment but is too polite to take up space. Set it in a distressed grunge face and it becomes aggressive, which breaks the register entirely; meh is not angry. The choice here — heavy, clean, lowercase, white on dark — is the one that reads as flat affect rather than hostility. Lowercase does a lot of that work. Capital Meh. would be a verdict. Lowercase meh. is a weather report.

I have spent a lot of my working life around status messages, and the analogy is not as forced as it looks. A monitoring system that reports OK for everything is lying. A system that pages you for every blip is useless. The useful signal is the one that distinguishes fine from degraded but not actionable, and the honest label for the second state is not a green checkmark. It is a shrug with a timestamp. That is roughly what this shirt is doing at the human scale: it is a status indicator for a person who is present, functional, and not currently interested in being enthusiastic about it.

What I like about the design as an object is that it does not overreach. It makes no promise. It does not claim the wearer is smart, or tired in an interesting way, or secretly brilliant. It does not ask you to agree with anything. It is the rare piece of printed text that can be worn into a room full of strangers without creating an obligation to explain itself. If someone asks what it means, the answer is the shirt. That is a complete thought.

The listing is the Meh Funny Sarcastic Saying Introvert Dry Humor shirt from Quirky Cotton Collective, and the visual observation above is the basis for everything I have said about the artwork. I am not going to tell you how it fits, because I have not worn it, and the preview does not show a body. I am not going to tell you it is soft, because the preview does not show a fabric. What the preview shows is the design, and the design is legible, deliberate, and correctly weighted for the word it carries.

The one thing I would flag for anyone reading the artwork closely: the period is not decoration. Without it, meh floats as an adjective waiting for a noun. With it, the word becomes a complete utterance — a sentence that has decided it has nothing further to add. That is a small typographic decision with an outsized effect on how the shirt reads, and it is the difference between a design that is merely quiet and one that is finished.

There is a broader point here about terse messages and the people who send them. In engineering culture, the shortest message is usually the most trusted one, because it implies the sender has already done the work of deciding what matters. Long explanations are for situations that are still unresolved. A one-word status is what you send when the situation is closed. meh. is that register applied to a mood rather than a system, and it lands for the same reason: it sounds like someone who has already finished thinking about it.

I do not think the shirt is making an argument about introversion, despite the listing title. The artwork does not show a person avoiding a party or a battery icon at 4%. It shows a word. If you want to read introversion into it, the design will not stop you, but it will not help you either. The stronger reading, and the one the typography actually supports, is simpler: this is a shirt for someone who is done being asked to have a take. That is a real and recurring condition, and it does not require a diagnosis to be worth naming.

The design also survives the test that kills most slogan shirts, which is the second read. The first time you see it, you get the joke. The second time, you notice the period, or the weight of the type, or the fact that the white is pure rather than off-white. There is not much there, but what is there is placed with intent. A shirt that only works once is a novelty. This one is closer to a piece of signage, and signage is supposed to keep working after the first glance.

If you are the kind of person who has ever written a one-line commit message that said more than the diff, the shirt is probably for you. If you are the kind of person who writes three paragraphs when one would do, it is probably a useful corrective. Either way, the artwork is what it is: a large, bold, lowercase word on a dark field, with a period at the end, declining to elaborate. I find that more honest than most of what gets printed on clothing, and I suspect the people who buy it do too.

The 33-Bit PTS Rollover at Hour 26.5: Captions, Sync, and the Wrap Nobody Tests

At 90 kHz, a 33-bit PTS counter wraps every 233 / 90,000 seconds. That is 95,443.7 seconds, or 26 hours, 30 minutes, 43.7 seconds. Most webcast reliability work happens in the first hour: encoder failover, CDN failover, caption reconnect. The wrap at hour 26.5 is rarely in the test plan, and it is where long-running contribution feeds, caption sidecars, and origin packagers quietly disagree about what time it is.

This article is about the mechanics of that wrap, the places it breaks, and the packet-level evidence that tells you which layer broke. It is not about a specific vendor. The behavior described here follows from the MPEG-2 systems timestamp field width, the RTP timestamp field width, and the way caption formats carry their own clocks.

What the 33-bit PTS actually is

MPEG-2 systems defines the presentation timestamp as a 33-bit value carried in the PES header, clocked at 90 kHz. The field is split across the PTS/DTS marker bits in the PES header, but the arithmetic is a plain 33-bit unsigned counter. The same 90 kHz clock drives the PCR in the transport stream adaptation field, which is a 42-bit value: 33 bits of 90 kHz base plus 9 bits of 27 MHz extension.

Two consequences follow directly from the field widths:

  • The PTS wraps at 233 = 8,589,934,592 ticks, which is 26:30:43.717 at 90 kHz.
  • The PCR base wraps at the same 33-bit boundary, but the PCR extension continues to count 27 MHz cycles, so the PCR as a whole wraps at 242 / 27,000,000 seconds, which is roughly 45.8 hours. The base and the extension do not wrap together.

That second point matters. A receiver that reconstructs time from PCR base alone will see a wrap at 26.5 hours. A receiver that reconstructs from the full 42-bit PCR will see a wrap at about 45.8 hours. If your packager and your caption engine use different reconstructions, they will disagree for a window in the middle of a long event.

Why hour 26.5 is the wrong number to test

If you test the wrap by running a 27-hour soak, you are testing one specific alignment: PTS starting near zero at t=0. In production, the PTS at the start of a contribution feed is whatever the encoder’s clock happened to be. A feed that starts at PTS 0x7FFF0000 will wrap within minutes. A feed that starts at PTS 0x00010000 will wrap after 26.5 hours.

The correct test is not a duration. It is a PTS offset. You want to start a feed with the encoder’s PTS initialized near 233 minus a few seconds, so the wrap happens inside a short test window. Most broadcast encoders expose this as a PTS offset or a “timecode start” setting. If yours does not, you can inject the condition at the packager by rewriting PTS values in the PES headers with a tool like tsp or a custom ffmpeg bitstream filter, but the cleanest test is at the encoder.

Where the wrap breaks: three failure classes

1. Caption timestamps that do not wrap

CEA-608 and CEA-708 captions are carried inside the video user data or as a separate caption service in the transport stream. The caption payload itself does not carry a 33-bit PTS. It carries a caption timing reference that is interpreted relative to the video PTS at the point of insertion. The caption engine, however, usually maintains its own internal clock, often derived from wall clock or from a monotonic counter.

When the video PTS wraps, the caption engine’s internal clock does not. If the caption engine computes caption display time as video_pts + caption_offset using a 64-bit accumulator, it will produce a caption time that is 233 ticks ahead of the video. The result is captions that appear roughly 26.5 hours late, or that are dropped entirely because the computed time is outside the player’s window.

WebVTT is a different case. WebVTT cues carry absolute timestamps in the form HH:MM:SS.mmm, as defined in the W3C WebVTT specification. There is no 33-bit field and no 90 kHz clock. A WebVTT file generated from a wrapped PTS will contain timestamps that jump backward by 26.5 hours at the wrap point. Players that seek within the file will handle this inconsistently: some will treat the backward jump as a discontinuity, some will clamp, and some will simply fail to display cues after the wrap.

The fix is to normalize caption timestamps to a monotonic timeline before they reach the player. That means the caption engine must be told the PTS wrap point, or must derive it from the video stream, and must add 233 to its accumulator at the wrap. This is a one-line change in most caption engines, but it requires that the engine know the wrap happened.

2. A/V sync drift across the wrap

Audio and video are separate elementary streams with separate PTS values. If both wrap at the same instant, the relative offset is preserved and sync is fine. If they wrap at different instants, or if one stream’s PTS is rewritten by a packager while the other is not, the offset changes by 233 ticks, which is 26.5 hours of audio-video offset. The player will either drop one stream or produce a permanent lip-sync error.

In practice, audio and video PTS wrap at the same instant only if they share a common clock and were initialized together. In a contribution encoder, they usually do. In a packager that remuxes from a contribution feed into HLS or DASH segments, the packager may rewrite PTS to start at zero for each segment. If the packager rewrites video PTS but not audio PTS, or vice versa, the wrap point differs between the two streams.

The diagnostic is straightforward. Extract PTS values from both elementary streams over a window that includes the wrap and plot the difference. A constant difference means sync is preserved. A step change of 233 ticks means one stream wrapped and the other did not.

3. Segment boundary and discontinuity handling

HLS and DASH both have explicit discontinuity signaling. RFC 8216 defines EXT-X-DISCONTINUITY for HLS, and the DASH specification defines a similar mechanism. A PTS wrap is a discontinuity in the timestamp domain, but it is not necessarily a discontinuity in the media. If the packager does not signal it, the player may attempt to interpolate across the wrap and produce a visible glitch or a stall.

The correct behavior is to signal a discontinuity at the segment boundary that contains the wrap, and to ensure that the segment’s internal timestamps are consistent. RFC 8216 states that each media segment must carry the continuation of the encoded bitstream from the previous segment, with timestamps and continuity counters continuing uninterrupted, except for segments explicitly signaled as discontinuities. A PTS wrap is exactly the case that requires the discontinuity tag.

If you are using fMP4 segments, the tfdt box carries the base media decode time. A wrap in the PTS domain must be reflected in the tfdt values, and the player must be told that the timeline is discontinuous. If the tfdt values simply continue past 233 while the PTS values wrap, the player will see a mismatch between the container timeline and the elementary stream timeline.

What the transport protocols do

RTP, as defined in RFC 3550, uses a 32-bit timestamp field with a media-specific clock rate. For video at 90 kHz, the RTP timestamp wraps every 232 / 90,000 seconds, which is about 13.25 hours. That is a different wrap point from the 33-bit PTS. If you are carrying MPEG-TS over RTP, you have two independent wrap points: the 33-bit PTS inside the TS payload and the 32-bit RTP timestamp in the RTP header.

RFC 3550 does not define a rollover mechanism for the RTP timestamp. The timestamp is intended to be used for relative timing within a session, and receivers are expected to handle wraparound by comparing timestamps modulo 232. The specification’s guidance on timestamp arithmetic is that receivers should use modular arithmetic and should not assume monotonicity across the entire session.

SRT and RIST carry timestamps in their own headers. SRT uses a 32-bit timestamp in the SRT data packet header, with a configurable clock rate. RIST uses a similar approach. Neither protocol defines a 33-bit PTS rollover, because neither carries MPEG-TS PTS directly. The PTS rollover is a property of the MPEG-TS payload, not of the transport.

WebRTC is a different case. WebRTC uses RTP timestamps, and the RTP timestamp is derived from the media clock. For video, the clock rate is typically 90 kHz, so the RTP timestamp wraps at about 13.25 hours. WebRTC receivers handle this with modular arithmetic. The 33-bit PTS rollover is not visible to WebRTC unless the WebRTC endpoint is carrying MPEG-TS, which is unusual.

QUIC, as defined in RFC 9000, does not carry media timestamps. It is a transport protocol for streams of bytes. If you are carrying media over QUIC, the media timestamps are inside the payload, and the QUIC layer does not interpret them. The 33-bit PTS rollover is therefore a payload-layer concern, not a QUIC concern.

Packet-level forensics: how to find the wrap

The first step is to capture the contribution feed at the point where the wrap occurs. If you are using SRT or RIST, capture at the receiver. If you are using MPEG-TS over UDP, capture at the packager input. The capture should include at least 30 seconds before and after the wrap.

Extract PTS values from the PES headers. With tsp, you can use tsp -I pcap -P pcr -P pes -O file to dump PCR and PTS values. With ffprobe, you can use ffprobe -show_packets -select_streams v to get packet timestamps, but note that ffprobe reports timestamps in the container’s timebase, which may already have been normalized by the demuxer. For raw PES analysis, tsp or a custom parser is more reliable.

Plot the PTS values over time. A wrap will appear as a sudden drop of approximately 233 ticks. If the drop is exactly 233, the stream is behaving correctly and the wrap is the only discontinuity. If the drop is a different value, or if there are multiple drops, there is a timestamp rewrite happening somewhere in the chain.

For caption forensics, extract the caption payload and compare its timing reference to the video PTS. If the caption timing reference does not wrap when the video PTS wraps, the caption engine is the problem. If the caption timing reference wraps but the player does not display the captions, the problem is in the player’s handling of the discontinuity.

For A/V sync forensics, extract PTS values from both audio and video elementary streams and compute the difference. A step change of 233 ticks at the wrap point indicates that one stream wrapped and the other did not. A gradual drift indicates a clock rate mismatch, which is a different problem.

Mitigation: a decision table

The right mitigation depends on where the wrap is handled. The following table summarizes the conditions and the corresponding action.

Condition Action
Encoder PTS initialized near zero, event shorter than 26.5 hours No action required. The wrap will not occur during the event.
Encoder PTS initialized near zero, event longer than 26.5 hours Configure the encoder to initialize PTS at a value that places the wrap outside the event window, or ensure the packager handles the wrap.
Encoder PTS initialized at an arbitrary value, event crosses the wrap Ensure the packager signals a discontinuity at the wrap and that the caption engine normalizes timestamps.
Caption engine uses wall clock or monotonic clock Normalize caption timestamps to the video PTS timeline, or configure the caption engine with the PTS wrap point.
Packager rewrites PTS to start at zero per segment Ensure the rewrite is consistent across audio and video, and that the discontinuity is signaled.
Player does not handle discontinuity Test the player with a synthetic discontinuity. If it fails, use a player that handles it, or avoid the wrap by re-initializing PTS.

Testing the wrap without waiting 26.5 hours

The most practical test is to generate a synthetic stream with a PTS that wraps within a few seconds. You can do this with ffmpeg by using the setts bitstream filter or by generating a raw PES stream and rewriting the PTS values. The following approach works with ffmpeg:

ffmpeg -f lavfi -i testsrc2=size=1280x720:rate=30 -f lavfi -i sine=frequency=1000:sample_rate=48000 -t 10 -c:v libx264 -c:a aac -muxdelay 0 -muxpreload 0 -output_ts_offset 95443 -f mpegts test.ts

The -output_ts_offset flag sets the initial PTS offset in seconds. Setting it to 95443 places the initial PTS near the wrap point, so the wrap occurs within the 10-second test. You can then inspect the output with tsp or ffprobe to see how the wrap is handled.

For caption testing, generate a WebVTT file with timestamps that cross the wrap point and mux it with the video. Then play the result in the players you support and observe whether the captions display correctly after the wrap.

For A/V sync testing, generate separate audio and video streams with different PTS offsets and mux them together. Then check whether the sync is preserved across the wrap.

What the standards do and do not say

MPEG-2 systems defines the 33-bit PTS field and the 90 kHz clock. It does not define a rollover mechanism. The expectation is that the PTS is a modulo-233 counter and that receivers handle the wrap. In practice, many receivers do not.

RFC 3550 defines the RTP timestamp as a 32-bit field with a media-specific clock rate. It does not define a rollover mechanism, but it does describe the timestamp as a sampling instant and expects receivers to use modular arithmetic. The RTP timestamp wrap is a separate event from the PTS wrap.

RFC 8216 defines the HLS discontinuity mechanism. It states that media segments must carry the continuation of the encoded bitstream, with timestamps continuing uninterrupted, except for segments explicitly signaled as discontinuities. A PTS wrap is a discontinuity in the timestamp domain, and the specification’s language supports signaling it as such.

The W3C WebVTT specification defines cue timestamps as absolute times in the form HH:MM:SS.mmm. It does not define a rollover mechanism, because WebVTT is not tied to a 33-bit clock. The implication is that WebVTT timestamps should be monotonic within a file, and that a backward jump is a discontinuity that the player must handle.

FAQ

Does the 33-bit PTS rollover affect all MPEG-TS streams?

Yes, if the stream runs long enough. The PTS is a 33-bit field, so it wraps every 26.5 hours regardless of the encoder or packager. Whether the wrap causes a visible problem depends on how the packager and player handle it.

Is the RTP timestamp rollover the same as the PTS rollover?

No. The RTP timestamp is a 32-bit field with a media-specific clock rate. For video at 90 kHz, it wraps every 13.25 hours. The PTS is a 33-bit field at 90 kHz, wrapping every 26.5 hours. They are independent counters and can wrap at different times.

Do SRT and RIST handle the PTS rollover?

SRT and RIST carry their own timestamps in their headers, but they do not interpret the MPEG-TS PTS. The PTS rollover is a payload-layer concern. SRT and RIST will deliver the packets correctly, but the packager or player must handle the PTS wrap.

How do I know if my caption engine handles the wrap?

Test it. Generate a stream with a PTS that wraps within a few seconds, add captions that cross the wrap point, and observe whether the captions display correctly. If they do not, the caption engine is not normalizing timestamps.

Can I avoid the wrap by re-initializing PTS?

Yes, if your encoder supports it. Re-initializing PTS at a value that places the wrap outside the event window avoids the problem entirely. This is the simplest mitigation for events shorter than 26.5 hours. For longer events, you must handle the wrap.

What is the best way to signal a PTS wrap in HLS?

Use EXT-X-DISCONTINUITY at the segment boundary that contains the wrap. Ensure that the segment’s internal timestamps are consistent and that the player is told that the timeline is discontinuous. RFC 8216 defines the tag and its semantics.

The takeaway

The 33-bit PTS rollover is a predictable event that most webcast pipelines are not tested for. The failure modes are caption timing errors, A/V sync drift, and segment boundary glitches. The fixes are known: normalize caption timestamps, signal discontinuities, and ensure that audio and video wrap together. The test is not a 27-hour soak; it is a synthetic stream with a PTS offset that places the wrap inside a short window. If you run long events, put the wrap in your test plan.

Measuring Lip-Sync Error in Production: Audio Cross-Correlation Against a Reference Track

Lip-sync error is one of the few production defects that viewers notice immediately and forgive slowly. In a webcast pipeline, it can be introduced at any stage: camera capture, contribution encoding, transport, origin packaging, CDN delivery, or the player’s decode and render path. The only way to fix it is to measure it. This article describes a practical method — audio cross-correlation against a reference track — and the packet-level evidence you need to interpret the result.

What the standards actually say

ITU-R BT.1359, Relative timing of sound and vision for broadcasting, is the primary international recommendation for acceptable audio-video timing error. It was approved in November 1998 and remains in force. The recommendation defines the relative timing of sound and vision for broadcasting and is the reference point most broadcast engineers cite when setting tolerances.

ITU-R BT.500, Methodologies for the subjective assessment of the quality of television images, is the companion document for subjective testing. The current version is BT.500-15, approved in May 2023. It is not a lip-sync measurement method, but it defines the viewing conditions and grading scales you need if you want to correlate objective measurements with human perception.

These are the two ITU-R recommendations that matter for this problem. BT.1359 gives you the target; BT.500 gives you the method for validating that your target is perceptually correct.

Why cross-correlation works

Cross-correlation measures the similarity between two signals as a function of the lag applied to one of them. If you have a reference audio track and a captured audio track from the same source, the lag at which the cross-correlation peaks is the delay between them. If you also have a video signal with a known visual event — a clap, a flash, a timecode burn-in — you can compare the audio delay to the video delay and compute the relative offset.

The method is not new. It is the basis of most automated lip-sync measurement tools. What matters in production is the reference signal and the measurement window.

Choosing a reference track

The reference track must be:

  • Time-aligned with the video at the point of capture. If the reference is generated separately from the camera, you need a common clock or a clap event to establish the offset.
  • Broadband and non-repetitive. Speech is ideal because it has a wide spectrum and no periodic structure that would create ambiguous correlation peaks. Music with a strong beat is worse because it produces multiple peaks at beat intervals.
  • Present in the final output. If the reference track is replaced or mixed with other audio, the correlation will degrade.

A spoken-word count with a visual clap at the start is the simplest reliable reference. For automated measurement, a continuous speech track with a known timecode and a visual timecode burn-in is better because it lets you measure drift over time, not just a single offset.

The measurement procedure

At a high level:

  1. Capture the reference audio and the reference video at the source.
  2. Capture the output audio and output video at the point you want to measure — typically the player or a capture device at the edge.
  3. Align the two audio tracks using cross-correlation to find the audio delay.
  4. Align the two video tracks using a visual event or timecode to find the video delay.
  5. Subtract the video delay from the audio delay. The result is the lip-sync error.

The cross-correlation itself is straightforward. In practice, you need to window the signals, normalize them, and handle the fact that the output audio may be compressed, resampled, or mixed with other content.

What can go wrong

Cross-correlation is robust to linear distortion but not to nonlinear processing. The following conditions will degrade or invalidate the measurement:

  • Lossy compression. AAC, Opus, and MP3 introduce phase and amplitude changes that reduce correlation peak sharpness. At typical streaming bitrates the peak is still detectable, but the confidence interval widens.
  • Packet loss and concealment. If audio packets are lost and concealed, the output audio no longer matches the reference. The correlation peak may shift or disappear.
  • Adaptive bitrate switching. If the player switches between renditions with different audio encoding parameters, the delay can change mid-stream. A single correlation measurement will not capture this.
  • Sample rate conversion. Resampling changes the time base. If the reference and output are at different sample rates, you must resample one to match the other before correlation.
  • Audio mixing. If the reference track is mixed with music or effects, the correlation is against a composite signal. The peak may still be present but weaker.

For each of these, the decision is the same: measure at a point in the pipeline where the audio is still uncompressed and unmixed, or accept a wider tolerance and validate with a subjective test.

Packet-level evidence

Cross-correlation gives you a number. Packet captures tell you why. The relevant metrics are:

  • RTP timestamp deltas. If the audio and video RTP streams are synchronized, their timestamps should advance at the same rate relative to a common clock. A drift in the delta between audio and video timestamps is a direct indicator of lip-sync error.
  • Arrival time jitter. If audio and video packets arrive with different jitter profiles, the playout buffer may introduce different delays. This is common when audio and video take different paths through the network.
  • Encoder and decoder timestamps. PTS and DTS values in the transport stream or container tell you what the encoder intended. If the player ignores them or applies its own offset, the error is in the player, not the network.
  • Packet loss and retransmission. SRT and RIST both have retransmission mechanisms. If audio and video have different loss profiles, the retransmission delays will differ.

In a production webcast, the most common cause of lip-sync error is not the network. It is the encoder or the player. Contribution encoders sometimes apply different buffering to audio and video. Players sometimes apply audio delay to compensate for video decode latency, and that compensation can be wrong.

Transport protocols and their effect on sync

SRT, RIST, WebRTC, and QUIC all have different mechanisms for handling timing and loss. None of them guarantee lip-sync by themselves. They guarantee delivery; sync is the responsibility of the encoder and decoder.

  • SRT uses a timestamp-based retransmission mechanism. It can deliver audio and video with low jitter, but if the sender does not preserve the relative timing of the two streams, the receiver cannot reconstruct it.
  • RIST has a similar approach with different retransmission profiles. The same caveat applies.
  • WebRTC uses RTP and RTCP. It has a built-in mechanism for lip-sync: the RTP timestamp and the RTCP sender reports allow the receiver to align audio and video. If the sender does not populate these correctly, the receiver cannot align them.
  • QUIC is a transport protocol, not a media protocol. It provides streams and reliability, but it does not define how audio and video should be synchronized. That is left to the application.

The practical implication: if you are measuring lip-sync error, you need to know which protocol is in use and whether it preserves the timing information you need. In most cases, the timing information is present but not used correctly.

A decision table for measurement points

Measurement point What you can measure What you cannot measure
Camera output Baseline sync between audio and video at capture Anything downstream
Encoder input Sync after capture and before encoding Encoder-induced delay
Encoder output Encoder-induced delay and drift Network and player effects
Origin output Packaging and origin effects CDN and player effects
Player output End-to-end lip-sync error Which stage caused it

Measure at the player for the final number. Measure at the encoder output and origin output to isolate the cause.

Limitations and when to use subjective testing

Cross-correlation is an objective measurement. It tells you the delay between two signals. It does not tell you whether a viewer will notice. ITU-R BT.500 defines the subjective methods for assessing quality, and while it is focused on video, the same principles apply to audio-video sync: controlled viewing conditions, a panel of viewers, and a grading scale.

If your objective measurement says the error is within tolerance but viewers complain, the problem may be that your tolerance is wrong for the content. Fast speech, close-up shots, and percussive sounds make lip-sync errors more noticeable. Wide shots and ambient audio make them less noticeable.

The practical approach is to use cross-correlation for continuous monitoring and BT.500-style subjective testing for validation. If the objective measurement is stable and within BT.1359 limits, and subjective testing confirms it, you have a defensible position.

FAQ

What is the acceptable lip-sync error for broadcast?

ITU-R BT.1359 defines the relative timing of sound and vision for broadcasting. The exact limits are in the recommendation, which is available from the ITU. For production webcasts, the same limits are a reasonable starting point, but you should validate against your own content and audience.

Can I measure lip-sync error without a reference track?

Yes, but it is harder. You can use visual events — a clap, a flash, a mouth opening — and measure the audio delay relative to those events. The accuracy depends on the precision of the event detection. A reference track is more reliable because it gives you a continuous signal to correlate against.

How often should I measure lip-sync error?

For a live production, continuous monitoring is ideal. If that is not possible, measure at the start of each session and after any change to the encoder, transport, or player configuration. Drift can occur over long sessions, so a periodic check is useful.

What tools can I use?

Any audio analysis tool that supports cross-correlation can be used. For packet-level analysis, Wireshark with RTP and SRT dissectors is standard. For automated measurement, you can write a script using a library like FFmpeg for decoding and NumPy or similar for correlation.

Does adaptive bitrate switching affect lip-sync?

It can. If the player switches between renditions with different audio encoding parameters or different buffering, the audio delay can change. This is one of the cases where a single correlation measurement is not enough; you need to measure continuously or at least at each switch.

References

NetEQ’s Accelerate: How Audio Time-Stretching Drifts Lip-Sync Over a Long WebRTC Session

Lip-sync drift in a long WebRTC session is rarely a single catastrophic event. It is usually the slow accumulation of small timeline corrections that the audio pipeline makes to keep its jitter buffer healthy. NetEQ’s Accelerate operation is one of those corrections. Understanding what it does to the audio playout timeline — and what it does not do — is the difference between chasing a phantom codec bug and fixing the actual clock relationship between audio and video.

What NetEQ’s Accelerate actually changes

NetEQ is the adaptive jitter buffer and decoder inside WebRTC’s audio path. Its source tree in the WebRTC repository includes accelerate.cc, preemptive_expand.cc, time_stretch.cc, expand.cc, merge.cc, and decision_logic.cc — the components that decide when to compress or stretch audio to maintain buffer depth. The Accelerate operation shortens the audio signal in the time domain. It does not drop packets and it does not change RTP timestamps on the wire. It changes how many output samples are produced for a given span of decoded input.

The practical consequence: after an accelerate event, the audio renderer has consumed more RTP timestamp duration than the wall-clock time it spent playing. The audio stream’s playout position relative to its own RTP timeline has moved forward. If the video renderer is still pacing against its own RTP timeline and the shared RTCP-derived reference, the two media clocks are now offset by the amount of time removed.

This is not a bug in NetEQ. It is the intended mechanism for recovering buffer depth after jitter or packet loss. The question is whether anything in the WebRTC stack feeds that offset back into video playout.

Why audio and video do not share a playout clock

RFC 3550 specifies that audio and video are transmitted as separate RTP sessions, with separate SSRCs, separate sequence number spaces, and separate timestamp clocks. Synchronized playback is achieved using timing information carried in RTCP packets for both sessions — specifically the NTP timestamp and RTP timestamp pair in RTCP Sender Reports. The RFC is explicit that there is no direct coupling at the RTP level between the audio and video sessions.

That architecture means the receiver must reconstruct a common timeline from two independent RTP timestamp spaces. In WebRTC, this is done by mapping each stream’s RTP timestamps to a common capture clock using the RTCP SR pairs, then scheduling playout against that common clock. The mapping is established at the start of the session and updated as new Sender Reports arrive.

NetEQ’s accelerate operation does not alter the RTP timestamps in the packets it processes. It alters the relationship between decoded sample count and output sample count. From the perspective of the RTP timestamp mapping, the audio stream is still where it always was. From the perspective of the audio device, the stream has moved forward. That gap is the drift.

The feedback question

Does WebRTC adjust video playout when NetEQ accelerates audio? The short answer from the specifications is: not through any mechanism defined in RFC 3550 or RFC 8825. The WebRTC overview (RFC 8825) describes the protocol suite but does not define a cross-media playout correction loop driven by audio jitter buffer operations. The W3C WebRTC Stats specification exposes metrics that can reveal the behavior — jitterBufferDelay, jitterBufferEmittedCount, concealedSamples, concealmentEvents, and totalSamplesReceived on RTCInboundRtpStreamStats — but exposing a metric is not the same as closing a control loop.

In practice, implementations may apply their own A/V sync logic. The specifications do not mandate one, and the behavior is implementation-specific. That is the honest answer, and it is why the drift is worth measuring rather than assuming.

What the stats can tell you

The W3C WebRTC Stats document defines cumulative counters that are designed to be sampled twice and differenced. The relevant ones for this problem:

  • jitterBufferDelay — total time samples have spent in the jitter buffer, in seconds. Divide by jitterBufferEmittedCount to get average buffer delay.
  • jitterBufferEmittedCount — total number of samples emitted from the jitter buffer.
  • concealedSamples — samples generated by concealment (packet loss concealment or expand operations).
  • concealmentEvents — number of concealment events.
  • totalSamplesReceived — total samples received.

These counters do not directly expose accelerate events. The NetEQ source tree includes statistics_calculator.cc and neteq_network_stats_unittest.cc, which track internal operations, but the W3C stats surface does not currently define an acceleratedSamples counter. That means you cannot read accelerate volume directly from getStats(). You can infer it: if jitterBufferDelay stays low while concealedSamples and concealmentEvents climb, the buffer is being managed aggressively. If the audio playout position drifts relative to video, accelerate is a candidate contributor.

A practical measurement approach: sample getStats() every 10 seconds, compute the delta in jitterBufferDelay divided by the delta in jitterBufferEmittedCount, and log it alongside the observed A/V offset. If the average buffer delay is stable but the offset grows, the cause is more likely clock skew between the audio and video capture devices than NetEQ operations.

Clock skew versus NetEQ acceleration

RFC 7273 addresses this directly. It notes that RTP implementations typically assume NTP timestamps are taken using unsynchronised clocks and must compensate for absolute time differences and rate differences. Without a shared reference clock, RTP can time-align flows from the same source at a given receiver using relative timing, but tight synchronization between different receivers or between different senders is not possible.

In a WebRTC session, the audio and video capture devices on the sender may have independent crystal oscillators. A 100 ppm difference between two clocks produces 0.1 ms of drift per second, or 360 ms per hour. That is enough to become visible. NetEQ’s accelerate operation adds to this only when it fires. The two mechanisms are distinct:

Mechanism Effect on audio playout timeline Rate Detectable via
Clock skew (sender audio vs. video capture) Continuous, monotonic offset Proportional to ppm difference RTCP SR NTP/RTP pairs over time
NetEQ Accelerate Stepwise forward jump in audio playout position Event-driven, tied to jitter/loss Indirect via jitterBufferDelay and concealment counters
NetEQ Expand/PLC Stepwise backward or hold in audio playout position Event-driven concealedSamples, concealmentEvents
Video frame drop/repeat Stepwise change in video playout position Event-driven framesDropped, framesDecoded

The decision table matters because the mitigation differs. Clock skew requires a clock discipline mechanism or periodic resynchronization. NetEQ acceleration requires either accepting the offset or implementing a cross-media correction that the specifications do not define.

What you can actually do

There is no standard WebRTC API to disable NetEQ’s accelerate operation. The decision logic in decision_logic.cc and the buffer level filter in buffer_level_filter.cc determine when it fires. What you can control:

  1. Reduce the conditions that trigger accelerate. Accelerate fires when the jitter buffer is deeper than target. A more stable network path — lower jitter, fewer loss bursts — means fewer accelerate events. If you control the contribution encoder, enabling forward error correction or retransmission can reduce the loss that drives buffer growth.
  2. Measure the offset directly. Do not rely on getStats() alone. Capture the audio and video render times at the receiver and compute the offset. If you have access to the sender, compare the RTCP SR NTP timestamps for audio and video to detect clock skew at the source.
  3. Resynchronize periodically if your application allows it. Some conferencing applications insert a brief silence or a video freeze to re-anchor A/V sync. This is application-level, not protocol-level, and it is audible or visible. It is a trade-off, not a free fix.
  4. Check whether your implementation already corrects. Some WebRTC implementations apply a slow audio resampler correction to track the video clock. If yours does, the drift may be bounded. If it does not, the drift is unbounded over a long session.

What the specifications do not say

It is worth being precise about the limits of the sourced material. RFC 3550 defines the RTP and RTCP mechanisms for synchronization. RFC 8825 describes the WebRTC protocol suite. The W3C WebRTC Stats document defines the metrics. None of these documents specify a maximum accelerate ratio, a maximum cumulative offset, or a required cross-media correction loop. The NetEQ source tree shows the components — accelerate.cc, time_stretch.cc, preemptive_expand.cc — but the source code is the implementation, not a specification of behavior.

That means any claim about a specific drift threshold or a specific session duration after which lip-sync becomes perceptible is not supported by the primary sources. The honest position: measure your own pipeline. The mechanisms are known. The thresholds are not universal.

FAQ

Does NetEQ’s Accelerate change RTP timestamps?

No. Accelerate operates on decoded audio samples in the time domain. The RTP timestamps in the packets are unchanged. The effect is on the relationship between decoded sample count and output sample count, which shifts the audio playout position relative to the RTP timeline.

Is there a getStats() metric for accelerated samples?

Not in the current W3C WebRTC Stats specification. The available counters include jitterBufferDelay, jitterBufferEmittedCount, concealedSamples, and concealmentEvents. Accelerate events must be inferred from buffer behavior rather than read directly.

Can I disable Accelerate?

There is no standard API to disable it. The decision logic is internal to NetEQ. You can reduce the conditions that trigger it by improving network stability and reducing packet loss, but you cannot turn it off through a WebRTC API.

Is lip-sync drift more likely from clock skew or from NetEQ?

Clock skew is continuous and proportional to the ppm difference between capture clocks. NetEQ acceleration is event-driven and tied to jitter buffer management. Over a long session with a stable network, clock skew is the more likely dominant contributor. Over a session with bursty loss and deep jitter buffers, NetEQ operations contribute more.

How do I measure A/V offset in a WebRTC session?

Sample getStats() at known intervals, compute the delta in jitterBufferDelay divided by the delta in jitterBufferEmittedCount, and log it alongside the observed A/V offset. If you have access to RTCP Sender Reports, compare the NTP timestamp and RTP timestamp pairs for audio and video to detect clock skew at the source.

Sources