Roku Developer Program

Developers and content creators—a complete solution for growing an audience directly.
cancel
Showing results for 
Search instead for 
Did you mean: 
mkammes
Level 8

~2 seconds lost on front of HLS stream, delaying captions

Hello all:

I've noticed that when I encode HLS files for my channel, the first ~2 seconds never play....first frame of action is 2 seconds into the video. I noticed that this also throws off the captions (sidecar SRT), too...they play ~2 seconds later out of sync (late).

I tried remedying this by encoding 2 seconds of black before the video begins, and retimed the captions. While this works visually - e.g. the video plays fully - the captions are the same amount off.

Is something odd about how the HLS streams are buffered and played compared to the CC files?

FWIW, SRT and HLS streams are housed on the same S3 cloud.

Channel is live if anyone wants to see - use the code 5THINGS (public channel) or https://channelstore.roku.com/details/84411/5-things .

Thanks!

~Michael
0 Kudos
11 Replies
mkammes
Level 8

Re: ~2 seconds lost on front of HLS stream, delaying caption

Hate to *bump*, but hoping for some info!
0 Kudos
Highlighted
mkammes
Level 8

Re: ~2 seconds lost on front of HLS stream, delaying caption

I posted this question on reddit, and got an answer. I'm sharing it here in case anyone else come across sync issues:

"I'm guessing the issue is that the Roku is waiting for a key frame in order to start playing the video. IIRC, Roku recommends that every segment begins with a key frame (sometimes called i-frames). If the first segment is missing one till 2 seconds in it may explain the issue. I'd check your encoder settings and see if you can figure out how to make that change."

I looked at my encoder settings (Sorenson Squeeze 10), and indeed, my keyframes are set to every 2 seconds. This seems like too much of a coincidence to be wrong.

I'm currently investigating how to force a keyframe on the first frame of the file.

Hope this helps others who are searching.
0 Kudos
EnTerr
Level 8

Re: ~2 seconds lost on front of HLS stream, delaying caption

Hmm, a file necessarily has to start with "i" (intra) frame, it can't start with "p" (predicted) or "b" (bi) now, can it - since they are based on at least one previous anchor frame.

Do couple of quick checks, e.g. if you set every frame to be keyframe (huge file but never mind for now) - does delay disappear? Also if you set from 2 to 4 sec - does the OOS increase?

See also if "Auto Key Frame on Scene Change" is checked (from random search saw https://support.sorensonmedia.com/custo ... tomization )
0 Kudos
mkammes
Level 8

Re: ~2 seconds lost on front of HLS stream, delaying caption

Thanks for the input. I, too, came across this note.

Those options are not available when dealing with HLS streams (ts - segmented files). If was doing standalone MP4s - yes.

I can certainly adjust the keyframe interval....1 second is the shortest delay I can set, so I could try 1, and then bump it up a few to see if there is any difference.

Any other thoughts?
0 Kudos
belltown
Level 7

Re: ~2 seconds lost on front of HLS stream, delaying caption

It would be unusual for your encoder not to start with a keyframe. You might want to verify whether or not your video does indeed start with a keyframe. A tool such as ffprobe should be able to help with that.
https://github.com/belltown/
0 Kudos
mkammes
Level 8

Re: ~2 seconds lost on front of HLS stream, delaying caption

What's interesting is that while examining the structure, I came across this via Media InfoSmiley Saddebug, complete) (I'll spare you the entire read out)

This is of the first fragment:

Duration : 00:00:05.839
Delay : 2000.000000
Delay : 2s 0ms
Delay : 2s 0ms
Delay : 2s 0ms
Delay : 00:00:02.000


So, there is a delay in there for some reason, and the 2 seconds matches the sync offset. So, perhaps it's not a keyframe issue, but an inherent delay in the stream. Not sure how to remedy this, short of moving to another transcoder where I can specify this manually (via CLI, a la ffmpeg, or an enterprise encoder).

ffprobe doesn't seem to be displaying the keyframe pattern (I,P,B) , but perhaps I'm just not using the switches correctly. However, if the assumption is that most transcoders start with a keyframe, then perhaps I should switch the investigation to the delay that Media Info finds.
0 Kudos
belltown
Level 7

Re: ~2 seconds lost on front of HLS stream, delaying caption

"mkammes" wrote:

ffprobe doesn't seem to be displaying the keyframe pattern (I,P,B) , but perhaps I'm just not using the switches correctly.

Here's an example:


c:\junk>ffprobe junk-1.ts -hide_banner -select_streams v -show_frames -show_entries frame=pict_type -print_format csv

Input #0, mpegts, from 'junk-1.ts':
Duration: 00:00:02.09, start: 0.030756, bitrate: 5173 kb/s
Program 1
Metadata:
service_name : Service01
service_provider: FFmpeg
Stream #0:0[0x100]: Video: mpeg2video (Main) ([2][0][0][0] / 0x0002), yuv420p(tv), 1920x858 [SAR 1:1 DAR 320:143], 24 fps, 24 tbr, 90k tbn, 48 tbc
Stream #0:1[0x101]: Audio: mp2 ([3][0][0][0] / 0x0003), 44100 Hz, stereo, s16p, 128 kb/s
frame,Iside_data,GOP timecode,8,00:00:00:00

frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,Pside_data,GOP timecode,8,00:00:00:12

frame,I
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,Pside_data,GOP timecode,8,00:00:01:00

frame,I
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,Pside_data,GOP timecode,8,00:00:01:12

frame,I
frame,Pside_data,GOP timecode,8,00:00:01:14

frame,I
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P

c:\junk>
https://github.com/belltown/
0 Kudos
mkammes
Level 8

Re: ~2 seconds lost on front of HLS stream, delaying caption

Many thanks for the syntax.

It does appear there is a keyframe on the first frame, so apparently this is not the smoking gun. Any thoughts on the start delay I found via Media Info?


Input #0, mpegts, from '/Users/xxxxx/Desktop/hls test/frag-1.ts':
Duration: 00:00:06.01, start: 2.000000, bitrate: 664 kb/s
Program 1
Metadata:
service_name : Service01
service_provider: FFmpeg
Stream #0:0[0x100]: Video: h264 (Baseline) ([27][0][0][0] / 0x001B), yuv420p(tv), 384x216 [SAR 1:1 DAR 16:9], 23.98 fps, 23.98 tbr, 90k tbn, 47.95 tbc
Stream #0:1[0x101]: Audio: aac (LC) ([15][0][0][0] / 0x000F), 44100 Hz, stereo, fltp, 118 kb/s
frame,I
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,I
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,I
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
frame,P
0 Kudos
belltown
Level 7

Re: ~2 seconds lost on front of HLS stream, delaying caption

"mkammes" wrote:
Many thanks for the syntax.

It does appear there is a keyframe on the first frame, so apparently this is not the smoking gun. Any thoughts on the start delay I found via Media Info?

No idea, I'm afraid. I would try a different encoder, if you have that option. Perhaps one of the HLS experts from Roku may be able to help.
https://github.com/belltown/
0 Kudos