Roku Developer Program

[nmcconnell]

During playback(Roku2XD) of a single-rate HLS stream I create with FFmpeg, sync disruptions, or hiccups, can be observed at certain segment transitions. They occur with video using main or baseline profile, at various resolutions. These hiccups do not occur if the TS file is processed with Apple's File segmenter or HTTPSegmenter. These problems are not seen when playing the stream on iOS or Android(JellyBean). The hiccups are also not seen with a video-only stream.
The hiccup starts from few frames towards the end of one segment until few frames at the beginning of the next segment, so it is not after the transition to the new segment. It first skips a few video frames (around 30+) causing video to get ahead of audio, pauses at a video frame for audio to catchup before continuing smooth playback. For example, with segments 83 & 84, it skips directly from frame 25156 to 25166, 25166 to 25170, 25170 to 25177, 25177 to 25183 and 25183 to 25187. It then plays frames 25187 to 25204, pauses at 25204 for some time while audio catches up and continues smooth playback after that. 25173 is the last frame of segment 83 and the problem started at 25156 (half a second earlier).

I am restricting my report of an issue to a single stream, to make things simpler. I've also included frame-numbers burned into the video for better troubleshooting. The playback issues I've encountered are worse when multiple rates are available, using the same encoder/segmenter path. With multiple rates, I see that at some transitions, Roku going directly to the lowest rate stream.

I recorded what I see with a video camera, as words can only describe so much.
Demo of what it looks like:
http://realonemedia.s3.amazonaws.com/40bebfd938d1499db5cb228bd6c0c7e7/ef7b38211bcd4dad91716d05b2048b...

The problematic stream, using FFmpeg's -ssegment:
http://realone-int2.s3.amazonaws.com/881700293bfb42b2ad89f2417c883e7f/909eb504b6ab426d934684635bea52...

The same source stream, gone through HTTPSegmenter
http://realone-int2.s3.amazonaws.com/881700293bfb42b2ad89f2417c883e7f/909eb504b6ab426d934684635bea52...

The video-only version of the stream, using FFmpeg's -ssegment:
http://realone-int2.s3.amazonaws.com/881700293bfb42b2ad89f2417c883e7f/909eb504b6ab426d934684635bea52...

I encode/segment to HLS with FFmpeg using the following command parameters(In my case, the output came out as 960x540pp):

ffmpeg -i $inputFileNameFullPath -sn -pix_fmt yuv420p -c:v libx264 -b:v 1436k -maxrate 2154k -bufsize 2872k -threads 4 -preset faster -profile:v main -force_key_frames "expr:gte(t,n_forced*5)" -flags +cgop -c:a libvo_aacenc -b:a 64k -ac 2 -vsync vfr -f mpegts pipe:1 | ffmpeg -f mpegts -i pipe:0 -c copy -map 0 -f ssegment -segment_time 10 -segment_list $playlistName -segment_list_flags +live -segment_format mpegts -avoid_negative_ts 1 $segmentName

Here are some things I've tried, in an attempt to workaround or troubleshoot the issue:

1. Resampling audio from 44.1 to 48kHz
2. Changing frame-rate from 29.97 to 30.0fps with 48kHz audio
3. Setting adrift_threshold to 0.01
4. Using -f hls instead of -ssegment
5. Using FDK instead of VO AAC encoder
6. Writing TS file, then segmenting, instead of piping

An FFmpeg developer I've been in contact with suggested it might be a result of PTS interleaving not being perfect.

[RokuJoel]

Thanks, I'll see if our engineers can give any feedback, please leave those links live for a few days, or, if you don't want to do that, PM me some links that we can use for testing.

- Joel

[RokuJoel]

from one of our engineers:

I pulled one of the segments and noticed that it doesn't start with a PAT/PMT like it's supposed to. This could be part of the problem. The Roku is complaining that it's late. The PTS issue he referenced could certainly be happening. I pulled down the tstools and inspected his files, http://code.google.com/p/tstools/. This is something they can do to inspect this further comparing working vs not working.

Hope that helps.

- Joel

[nmcconnell]

Which segment doesn't start with PAT/PMT like it's supposed to?
How can I reproduce Roku complaining that it's late? What method of logging?

Thanks,
-Neal

[RokuJoel]

I believe he used tstools, referenced in the post I made, to test the segment. Which segment he picked, I have no idea, I would imagine the problem would exist in all of them if they were created from the same command line.

You could use roSystemLog which should give you any http errors, also roVideoPlayerEvent.getinfo()

When we say a segment is late, we mean http 404 messages, in other words, the device attempts to retrieve a segment and the segment doesn't exist yet, even though the segment URL is in the playlist.

- Joel

[nmcconnell]

The logs suggested do not contain TS parsing errors or warnings- just requests showing what segment files were loaded.

When I concatenate the segments encompassing a bad segment transition, and play the resuling TS file locally on Roku3, there is no hiccup. This tells me that there is nothing wrong with the packetization or audio/video bitstreams, but rather something about loading particular segments.

The segments from FFmpeg's segmenter satisfy Roku's initial requirements:
1) The first frame must be an I-frame
Actually, an IDR frame. Yes

2) The SPS must be set correctly.
They are the same throughout.

3) There must be a PAT and PMT (video and audio) message muxed into the segment before the actual video and audio PID content
This is true for all segments, including the ones that have hiccups. This cannot be said for the segments from HTTPSegmenter, which sometimes has audio preceding the PAT/PMT, but it still plays without hiccups.

Does Roku do any sort of pre-loading segments, and is this adjustable?
The Are there any extra requirements regarding TS PIDs, PCR frequency, when considering going from one segment to another?

Does anyone have a channel which encodes its content using FFmpeg with built-in segmentation (-f ssegment, or -f hls) ?

[RokuBradO]

The failed stream at the time of the frames is the result of a video underrun. There is a discrepancy in the PTS value we present to the decoder and what's actually returned.

[nmcconnell]

Questions, questions...

Is this a Roku bug?
Why is this not seen if the segments are concatenated and played locally?
Why is this not seen with the output from HTTPSegmenter, which is essentially just a splitter that knows video key-frames?
Is there a suggested way to work around said discrepancy in the PTS value? That is, if one were to tweak FFmpeg's MPEG-TS muxing/segmentation, what characteristics or computation would need to change?

Here is another, shorter(13 segments), stream that exhibits more frequent hiccups, with one instance giving an audio decoder chirp:
http://realonemedia.s3.amazonaws.com/25 ... ylist.m3u8

Here is the same stream, but with SDT and audio packets stripped out. It plays smoothly for me.
http://realonemedia.s3.amazonaws.com/25 ... ylist.m3u8

They have no DTS(only PTS) or B-Frames to worry about.