OTA HD and Lip Sync Problems

[Richard H.]

I've noticed this from time to time, especially on channel 36 but also on other stations where the voice or music is out of sync with the person's lip movements, singing or speaking, during HD telecasts. Has anyone else experienced this problem? What causes it?

Thanks in advance

[Gregg Lengling]

this is a common problem that will get better with time as stations/networks install newer/better equipment.  Seeing as all Audio and Video are digital now and the streams are basically separate there is a lot of work making sure both streams are in sync.  There was a very big article (about 3 months long) in DTV Magazine (Digital Television Technology).  I noticed this just recently on channel 36, but also last night on SciFi.

[John L]

Here I thought it was my cheap Hauppaughe Win-TV-D card.  I sometimes notice audio out of sync with the video.  

-John L.

[pdtricn]

I have found this to be more common now than in the past, maybe I am more sensitive to it. It does annoy me quite a bit. Anyone else really distracted by this? One source in a leading home theater magazine (recent issue) stated it was due to the audio signal being "faster" than the video signal. I don't know if that explains all of it as I see it occur on my non-HD Tv with analog sound as well. I wonder if it is coming from the networks or the cable signal (I have cable). My father in law has satellite and his is extremely noticeable. I thought there was a regulation on how long a delay there can be and I wonder if the networks are exceeding that.

[waterhead]

I do most of my HD viewing through a PC based card. When this happens, if I close out the TV viewer and restart it, it usually comes back OK.

I have noticed it with different brand cards and on differerent operating systems (Windows & Linux). It seems to be more common with SD material that is broadcast in digital format.

[foxeng]

Not living in Milwaukee I don't know how bad it is on your local stations, but I can tell you from reading and talking to many people from around the country, lip sync is a problem everyone is having as we transition to digital. As has been pointed out, it is because the audio and video take separate paths before being transmitted to your TV and it arrives at your TV at different times. And it is usually the video arriving later than the audio. This is usually caused by the video having to jump through more hoops than the audio and every time the video goes through a frame synchronizer or video effects box or crawl box or anything, the video is slightly slowed down. The audio doesn't normally go through as many devices and so doesn't slow down as much.

We have had this problem all through televisions history, but in analog it wasn't as noticable unless you hung A LOT Of stuff on the video line like multiple frame syncs and video effects boxes and triple or quad boxes all at the same time! In the digital world, all of this stuff is computers now and each has their own CPUs and therefore their own processor speeds and anything put them takes time and the time it takes is usually longer than in the analog world so what you would normally have in line in the analog world and people didn't notice, the same boxes in the digital world, you have a noticable problem.

To help combat this, many of the manufacturers have recently started adding audio synchronizes in their boxes to match the audio with the video to reduce the problem. I know at my own station, one of the first 3 questions about a new piece of equipment is "does it have an audio synchronizer?" Luckily there are other places in the chain that an audio delay can be added so if you don't have gear with a built in synchronizer, you can delay later, and hopefully by putting in and taking our different boxes at different times, the lip sync issue doesn't change minute to minute!

Of course adding all of these audio buffers, if you will, causes other problems, particularly for live talent because now if they listen through the program line, as they have been for years, they hear the delay and it messes with them so we have to do little tricks to keep the talent happy.

I know at my station, most of the time we have a pretty good handle on the lip sync issue after MUCH trial and error annnnnddddd ......................... bitching from the public, but there are some days that things go really wacky for lots of reasons and we look like a badly dubbed Asian horror flick from the 60's. But it is getting better.

Stations are not the only one to have this issue but cable as well. Many of the systems are now taking digital feeds and then breaking them down into video/audio for various reasons and introducing lip sync errors as well. As neat as digital is, it is no free ride for anyone. Most stations, if not all are aware of it and are fighting it as hard as they can. There are usually no simple answers or solutions most of the time when this happens. But that is what happens when you are on the cutting edge of new technology.

[tencom]

Don't broadcasters add time stamps to there digital stream to  supposely take care of video, audio timing differences? isn't that the purpose of time stamps.

[foxeng]

If by time stamp you are talking about PCR, yes we do, but the problem is pre-PCR. PCR is done in the MPEG2 encoder, after station processing. If the audio and video is out of time with each other before it hits the encoder, PCR only times the out of time video/audio to a new out of time standard. Classic, garbage in, garbage out.

[SRW1000]

Not living in Milwaukee I don't know how bad it is on your local stations, but I can tell you from reading and talking to many people from around the country, lip sync is a problem everyone is having as we transition to digital...

Thanks for the details, Foxeng.  I didn't realize you were still visiting our little local forum.  I see your posts all the time at the AVS Forum, and I know you used to drop by here quite frequently.

Your information is very much appreciated.

Scott

[foxeng]

I lurk almost daily! I still like to look and see how much crap my fellow FOX stations are getting!  With everyone at full power now, not much to comment on these days so I don't post much. Even my postings on AVS has slowed way down as well.

[tencom]

If by time stamp you are talking about PCR, yes we do, but the problem is pre-PCR. PCR is done in the MPEG2 encoder, after station processing. If the audio and video is out of time with each other before it hits the encoder, PCR only times the out of time video/audio to a new out of time standard. Classic, garbage in, garbage out.

Besides the Program Clock Reference, then what is the purpose of the Presentation Time Stamp and the Decode Time Stamp? Are you saying once a video/audio timing error occurs it is diffucult to correct. Isn't it possible to include sync data at the programs  origination point that even, at the receivers decoder could correct any timing errors between audio and video

[foxeng]

Besides the Program Clock Reference, then what is the purpose of the Presentation Time Stamp and the Decode Time Stamp? Are you saying once a video/audio timing error occurs it is diffucult to correct. Isn't it possible to include sync data at the programs  origination point that even, at the receivers decoder could correct any timing errors between audio and video

PTS and DTS are receive functions and are related to the PCR.

From the Zenith 8-VSB docs:
The Presentation Time Stamp (PTS) indicates the instant at which an access unit should be removed from the receiver buffer, instantaneously decoded, and  presented for display.  Any delay in a practical decoding or display process can be compensated for if it is fixed.  Therefore, if a practical decoder has a variable delay, it must include buffering to restore a constant delay at the output.  Note that the display itself may include a delay, for example, the time for the vertical scanning of a CRT to reach the middle of the picture.

PTS (or DTS) is entered in the bitstream at intervals not exceeding 700 mS.  ATSC further constrains PTS (or DTS) to be inserted at the beginning of each access unit.

and:

The Decode Time Stamp (DTS) indicates the time at which an access unit should be instantaneously removed from the receiver buffer and decoded.  It differs from the Presentation Time Stamp (PTS) only when picture reordering is used for B pictures.  If DTS is used, PTS must also be provided in the bit stream.

PTS (or DTS) is entered in the bitstream at intervals not exceeding 700 mS.  ATSC further constrains PTS (or DTS) to be inserted at the beginning of each access unit.

What we are talking about are timing errors in the studio. Video and audio is split into separate paths in the studio and take separate routes until they are combined back together in the MPEG2 encoder where PCR, PTS and DTS take over. If the error exists at the time of encoder entry, PCR, PTS nor DTS can correct it since it has no way to know the input is out of time, only that the data that is input at time frame 0 on the video/audio inputs come out at frame 0 at the receiver.

[tencom]

Thank you, for the clairification. Sounds like sloppy engineering practices, at the source is responsible.

[foxeng]

Sounds like sloppy engineering practices, at the source is responsible.

Not in all cases. Just a tremendous on the job learning curve for many.

[tencom]

Not in all cases. Just a tremendous on the job learning curve for many.

Is DTS/PTS optional with the broadcaster and not always employed? I know the P frame is transmited before the B frames and it's up to the receivers decoder to reorder the P and B frames. Do you know if the  P frames contain motion vectors like the B frames and does the B frames contain any video elements, or does  the B frames depend soley on the  P frames for any video updates. You sound like you understand the ATSC process and maybe helpful to others to explain a very complex process.