Roku Developer Program

Join our online forum to talk to Roku developers and fellow channel creators. Ask questions, share tips with the community, and find helpful resources.
cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
nmcconnell
Visitor

Plaback hiccups with FFmpeg-produced HLS

During playback(Roku2XD) of a single-rate HLS stream I create with FFmpeg, sync disruptions, or hiccups, can be observed at certain segment transitions. They occur with video using main or baseline profile, at various resolutions. These hiccups do not occur if the TS file is processed with Apple's File segmenter or HTTPSegmenter. These problems are not seen when playing the stream on iOS or Android(JellyBean). The hiccups are also not seen with a video-only stream.
The hiccup starts from few frames towards the end of one segment until few frames at the beginning of the next segment, so it is not after the transition to the new segment. It first skips a few video frames (around 30+) causing video to get ahead of audio, pauses at a video frame for audio to catchup before continuing smooth playback. For example, with segments 83 & 84, it skips directly from frame 25156 to 25166, 25166 to 25170, 25170 to 25177, 25177 to 25183 and 25183 to 25187. It then plays frames 25187 to 25204, pauses at 25204 for some time while audio catches up and continues smooth playback after that. 25173 is the last frame of segment 83 and the problem started at 25156 (half a second earlier).

I am restricting my report of an issue to a single stream, to make things simpler. I've also included frame-numbers burned into the video for better troubleshooting. The playback issues I've encountered are worse when multiple rates are available, using the same encoder/segmenter path. With multiple rates, I see that at some transitions, Roku going directly to the lowest rate stream.

I recorded what I see with a video camera, as words can only describe so much.
Demo of what it looks like:
http://realonemedia.s3.amazonaws.com/40bebfd938d1499db5cb228bd6c0c7e7/ef7b38211bcd4dad91716d05b2048b...

The problematic stream, using FFmpeg's -ssegment:
http://realone-int2.s3.amazonaws.com/881700293bfb42b2ad89f2417c883e7f/909eb504b6ab426d934684635bea52...

The same source stream, gone through HTTPSegmenter
http://realone-int2.s3.amazonaws.com/881700293bfb42b2ad89f2417c883e7f/909eb504b6ab426d934684635bea52...

The video-only version of the stream, using FFmpeg's -ssegment:
http://realone-int2.s3.amazonaws.com/881700293bfb42b2ad89f2417c883e7f/909eb504b6ab426d934684635bea52...

I encode/segment to HLS with FFmpeg using the following command parameters(In my case, the output came out as 960x540pp):
ffmpeg -i $inputFileNameFullPath -sn -pix_fmt yuv420p -c:v libx264 -b:v 1436k -maxrate 2154k -bufsize 2872k -threads 4 -preset faster -profile:v main -force_key_frames "expr:gte(t,n_forced*5)" -flags +cgop -c:a libvo_aacenc -b:a 64k -ac 2 -vsync vfr -f mpegts pipe:1 | ffmpeg -f mpegts -i pipe:0 -c copy -map 0 -f ssegment -segment_time 10 -segment_list $playlistName -segment_list_flags +live -segment_format mpegts -avoid_negative_ts 1 $segmentName


Here are some things I've tried, in an attempt to workaround or troubleshoot the issue:
    1. Resampling audio from 44.1 to 48kHz
    2. Changing frame-rate from 29.97 to 30.0fps with 48kHz audio
    3. Setting adrift_threshold to 0.01
    4. Using -f hls instead of -ssegment
    5. Using FDK instead of VO AAC encoder
    6. Writing TS file, then segmenting, instead of piping

An FFmpeg developer I've been in contact with suggested it might be a result of PTS interleaving not being perfect.
0 Kudos
7 REPLIES 7
RokuJoel
Binge Watcher

Re: Plaback hiccups with FFmpeg-produced HLS

Thanks, I'll see if our engineers can give any feedback, please leave those links live for a few days, or, if you don't want to do that, PM me some links that we can use for testing.

- Joel
0 Kudos
RokuJoel
Binge Watcher

Re: Plaback hiccups with FFmpeg-produced HLS

from one of our engineers:

I pulled one of the segments and noticed that it doesn't start with a PAT/PMT like it's supposed to. This could be part of the problem. The Roku is complaining that it's late. The PTS issue he referenced could certainly be happening. I pulled down the tstools and inspected his files, http://code.google.com/p/tstools/. This is something they can do to inspect this further comparing working vs not working.


Hope that helps.

- Joel
0 Kudos
nmcconnell
Visitor

Re: Plaback hiccups with FFmpeg-produced HLS

Which segment doesn't start with PAT/PMT like it's supposed to?
How can I reproduce Roku complaining that it's late? What method of logging?

Thanks,
-Neal
0 Kudos
RokuJoel
Binge Watcher

Re: Plaback hiccups with FFmpeg-produced HLS

I believe he used tstools, referenced in the post I made, to test the segment. Which segment he picked, I have no idea, I would imagine the problem would exist in all of them if they were created from the same command line.

You could use roSystemLog which should give you any http errors, also roVideoPlayerEvent.getinfo()

When we say a segment is late, we mean http 404 messages, in other words, the device attempts to retrieve a segment and the segment doesn't exist yet, even though the segment URL is in the playlist.

- Joel
0 Kudos
nmcconnell
Visitor

Re: Plaback hiccups with FFmpeg-produced HLS

The logs suggested do not contain TS parsing errors or warnings- just requests showing what segment files were loaded.

When I concatenate the segments encompassing a bad segment transition, and play the resuling TS file locally on Roku3, there is no hiccup. This tells me that there is nothing wrong with the packetization or audio/video bitstreams, but rather something about loading particular segments.

The segments from FFmpeg's segmenter satisfy Roku's initial requirements:
1) The first frame must be an I-frame
Actually, an IDR frame. Yes

2) The SPS must be set correctly.
They are the same throughout.

3) There must be a PAT and PMT (video and audio) message muxed into the segment before the actual video and audio PID content
This is true for all segments, including the ones that have hiccups. This cannot be said for the segments from HTTPSegmenter, which sometimes has audio preceding the PAT/PMT, but it still plays without hiccups.

Does Roku do any sort of pre-loading segments, and is this adjustable?
The Are there any extra requirements regarding TS PIDs, PCR frequency, when considering going from one segment to another?

Does anyone have a channel which encodes its content using FFmpeg with built-in segmentation (-f ssegment, or -f hls) ?
0 Kudos
RokuBradO
Visitor

Re: Plaback hiccups with FFmpeg-produced HLS

The failed stream at the time of the frames is the result of a video underrun. There is a discrepancy in the PTS value we present to the decoder and what's actually returned.
0 Kudos
nmcconnell
Visitor

Re: Plaback hiccups with FFmpeg-produced HLS

Questions, questions...

Is this a Roku bug?
Why is this not seen if the segments are concatenated and played locally?
Why is this not seen with the output from HTTPSegmenter, which is essentially just a splitter that knows video key-frames?
Is there a suggested way to work around said discrepancy in the PTS value? That is, if one were to tweak FFmpeg's MPEG-TS muxing/segmentation, what characteristics or computation would need to change?

Here is another, shorter(13 segments), stream that exhibits more frequent hiccups, with one instance giving an audio decoder chirp:
http://realonemedia.s3.amazonaws.com/25 ... ylist.m3u8

Here is the same stream, but with SDT and audio packets stripped out. It plays smoothly for me.
http://realonemedia.s3.amazonaws.com/25 ... ylist.m3u8

They have no DTS(only PTS) or B-Frames to worry about.
0 Kudos
Need Assistance?
Welcome to the Roku Community! Feel free to search our Community for answers or post your question to get help.

Become a Roku Streaming Expert!

Share your expertise, help fellow streamers, and unlock exclusive rewards as part of the Roku Community. Learn more.