Roku Developer Program

[bryankaiser]

Tommy,

Yeah, anything I generate with Handbrake has the buffering issue. I think that it's the process of going directly from the 5.1 ac3 file to a 2 channel aac. Letting BeSweet do the mixdown and then using the resulting wav as the source for the aac encode is what appears to actually make this all work. From what I can find, Handbrake and ffmpeg use the same engine under the hood to do the aac encoding (faac), so it makes sense that whatever problems ffmpeg has handbrake would have. Although, handbrake is doing a mixdown (ffmpeg doesn't), so it doesn't get to the aac encoder in exactly the same form as it came off the disk. I would suspect that you could use anything that could turn an ac3 file into a wav, but then the levels would be wrong, etc.

What I might try later is to do an encode with Handbrake, and then replace the audio with ffmpeg. I'd love to have Handbrake generate an aac at better than 160kb though, but it doesn't look like you can do that. Maybe with the command line, but not through the GUI.

What I'd really like to know is what's causing the issue in the first place. What is it about the ac3 -> aac conversion that is causing the problem? That way lies madness though, so I think I may have to leave it alone before I end up looking at raw streams in a hex editor. 🙂

But trust me, I'd love to find settings for something that would just let me click a button and have it work.

[TommyTheKid]

I was previously leaving it as default, actually I didn't even go to the Audio Tab till today. The default on OSX 10.6 HandBrake:

Source: (English AC3 5.1) <-- obviously dependent on title
Audio Codec: AAC (faac), AAC (CoreAudio), AC3 Passthru
Mix Down: Mono, Stereo, Dolby Surround, Dolby Pro Logic II, 6 channel discrete
Sample Rate: Auto (ranges from 22.05-48KHz)
Bit Rate: 128 (ranges from 64-320)
DRC: 0


Right now I have Mix Down set on stereo, which is taking a LOT longer to encode, hope its not a waste of time. might know more after lunch.

~tommy

"bryankaiser" wrote:
Tommy,

Yeah, anything I generate with Handbrake has the buffering issue. I think that it's the process of going directly from the 5.1 ac3 file to a 2 channel aac. Letting BeSweet do the mixdown and then using the resulting wav as the source for the aac encode is what appears to actually make this all work. From what I can find, Handbrake and ffmpeg use the same engine under the hood to do the aac encoding (faac), so it makes sense that whatever problems ffmpeg has handbrake would have. Although, handbrake is doing a mixdown (ffmpeg doesn't), so it doesn't get to the aac encoder in exactly the same form as it came off the disk. I would suspect that you could use anything that could turn an ac3 file into a wav, but then the levels would be wrong, etc.

What I might try later is to do an encode with Handbrake, and then replace the audio with ffmpeg. I'd love to have Handbrake generate an aac at better than 160kb though, but it doesn't look like you can do that. Maybe with the command line, but not through the GUI.

What I'd really like to know is what's causing the issue in the first place. What is it about the ac3 -> aac conversion that is causing the problem? That way lies madness though, so I think I may have to leave it alone before I end up looking at raw streams in a hex editor. 🙂

But trust me, I'd love to find settings for something that would just let me click a button and have it work.

[TommyTheKid]

I will have to try to do it the way you tried cause the HandBrake mixdown didn't help. It seemed to delay the buffering issue till maybe 1:15-1:20, but it still happened... even with an excessive buffer (bps instead of kpbs). I tried to download x264 video from another source, but its in a .mkv, so I guess I need to re-package it 😉

Tommy

[bryankaiser]

Ok, so there is still an issue even though you were using CoreAudio and not faac.

There is something you could try. I haven't actually gotten around to doing this myself, but I was going to try and re-encode the audio from what handbrake produced using ffmpeg. It would be something like this:

ffmpeg -i FILE_FROM_HANDBRAKE.mp4 -acodec aac -ab 160k -ac 2 -vcodec copy OUTPUT.mp4

This will take your file from handbrake, copy the video stream (no recompressing) and will re-encode the audio. You're encoding the audio twice, which isn't ideal, but if it works it might be worth it.

If you can get ahold of ffmpeg and are willing to try, I'd love to see if this works. It shouldn't take long to run because you're only compressing the audio. Just make sure your input and output filenames are different.

Also, I've found that with these problematic files you can just FF to the point where they start to buffer (so 1:15 - 1:20 for your example). I've watched a bandwidth monitor when the Roku was playing problematic files and the amount of bandwidth it consumes grows less and less as the file plays. Eventually it can't keep it's buffer full and you see the constant rebuffering. The Roku would start out pulling 7 to 8Mb/s when the video started playing and end up at 150Kb/s at the end. At that rate it was impossible for the Roku to keep its buffer full.

[renojim]

"bryankaiser" wrote:

Also, I've found that with these problematic files you can just FF to the point where they start to buffer (so 1:15 - 1:20 for your example). I've watched a bandwidth monitor when the Roku was playing problematic files and the amount of bandwidth it consumes grows less and less as the file plays. Eventually it can't keep it's buffer full and you see the constant rebuffering. The Roku would start out pulling 7 to 8Mb/s when the video started playing and end up at 150Kb/s at the end. At that rate it was impossible for the Roku to keep its buffer full.

Anyone have any more information on this? I've noticed the same thing as bryankaiser with bw used declining as time goes on. What I've been able to do is to re-encode the problem video at 29.97 fps. I'm no video expert, but I have an avi that was from a DVD rip (I didn't do the rip, so I don't know the details how how that was done) that was originally 23 fps. If I use HandBreak to encode it using the defaults, i.e.,

HandBreakCLI -i ProblemFile.avi -o ProblemFile.m4v -O

I get a file with the bw decline and rebuffering issues. If I encode that same file at 29.97 fps, i.e.,

HandBreakCLI -i ProblemFile.avi -o FixedFile.m4v -O -r 29.97

the file plays fine and the bw used looks as expected - bursts at a maximum rate followed by periods of no bw being used.

-JT

[bryankaiser]

JT,

I still haven't figured out exactly what the issue is, but I have two theories:

1) There is something wrong with the container. An mp4 is just a container for audio and video streams. In the case of the roku, we have an h.264 video stream and an AAC audio stream. Because of the encoding process (to encode the dvd), the original audio and video streams have differing lengths (fractions of a second). Something happens during the encoding process because of these differences and I *think* that this is where the issue lies.

2) There is something screwy with the framerates when pulling off of a DVD. Your test lends support to this. I've also found a couple of instances where the audio is out of sync with the video after encoding. I tracked this down to being a problem where the audio was playing at a different framerate than the video.

I've found two ways to deal with the issue and get the videos working correctly (mostly, I'll talk about this in a bit). If I tell ffmpeg (the encoder I'm using) to trim off the last few seconds of a video, before it has a chance to run into the end of either the audio or video stream, my videos will play correctly on the Roku. Also, if I use something like AviSynth which creates a "virtual" video and use that to encode, then those videos play correctly. The "ffmpeg trim" approach has worked in all but one instance. I had one DVD where the audio was out of sync and that's what lead me to try out AviSynth.

So, I have videos that now play correctly on the Roku, and the bandwidth consumed doesn't exhibit the same behavior as before. The only issue I have at this point is that if I create bif files (the files that allow you to use the trick play feature) for my videos, the videos will no longer continue to play. I'm working on getting a test case together for this and will hopefully have something together today or tomorrow. The truth is that we can't really debug this since we can't get to the software (or hardware) where the issues are actually occurring. Only the Roku folks can really debug this, so I'm going to try and provide a test case for them to use. I don't know that they will, but it's worth a shot.

Bryan

[renojim]

Bryan,

You have some interesting results. I have a limited set of video files that I can experiment with unless I start ripping DVDs. I can say that I've witnessed the problem on videos that have come from sources other than DVDs. I have some downloaded TV shows that when encoded using the HandBrake defaults will exhibit the buffering problem. In every case I've seen, and I admit I have a small data set, the videos with the problem have a frame rate of 23-25 fps. In all of these instances, I've been able to take the original avi and encode it to a 29.97 fps mp4 and it will play fine.

I hope Roku will figure out what the issue really is. I'm not a stickler for quality (I'll probably be the last person in the world to get an HDTV) and I don't mind just setting the frame rate to 29.97, but I would like to know what's going on because I know eventually I'll run into a video that can't be fixed that easily.

I can't help with the trick play issue. It's one of those things that's on my list, but who knows when I'll get around to it. You're right about us not being able to get to the hw/sw where the issue is. Believe me, I'd love to have a shot at tinkering with it!

-JT

[sumosays]

I've been reading through this thread and have been messing around with this same issue all weekend. I've had luck so far encoding to 29.97fps using Handbrake from DVD folders on my HD. I've only tried it with 1 movie so far, but will have a few more encoded tonight and we'll see how those turn out tomorrow.

I'm not super technical, so these observations may or may not be useful, but I've noticed that when the video first loads, there is much more of a hang at the beginning of the load bar when there is a rebuffering issue.

There are also a few videos which I encoded a while ago with an older version of Handbrake which seem to work fine and those are all encoded at 23.98 fps. Unfortunately I can't figure out what makes those videos any different than some of the other ones that don't work. Looking at the difference in the length of the audio and video tracks don't seem to offer any consistent differences.

Anyway, I thought I'd throw in my observations in case it helps anyone out there with the issue.

[TommyTheKid]

For the people ripping DVDs, what are you using? I was using HandBrake (v0.9.4 32bit) on Mac OSX 10.5.6, and of couse it uses "VLC" (v1.0.5 32bit) to "open" the DVD. I am also using the (default on mac) "CoreAudio" encoder for AAC, which they claim produces better quality. There is an option to turn on "playback debugging" but I haven't had a chance to see what that does yet. I have tried it on my current development Roku, and it puts up some information at the beginning 10-15 seconds about the bandwidth. Maybe it would say something when it got angry at 1hr+? Unfortunately, everything I am dealing with at the moment is very short.

The only way I can think of to do this would be to have a way to set a "debug level" on the roVideoScreen object, to have it output a LOT information to the debug console. Even then, I am not sure what we'll end up with.

Tommy

[em_lazardo]

"bryankaiser" wrote:
Ok, so there is still an issue even though you were using CoreAudio and not faac.

There is something you could try. I haven't actually gotten around to doing this myself, but I was going to try and re-encode the audio from what handbrake produced using ffmpeg. It would be something like this:

ffmpeg -i FILE_FROM_HANDBRAKE.mp4 -acodec aac -ab 160k -ac 2 -vcodec copy OUTPUT.mp4

Success w 23.976 fps thanks to the briankaiser hint, using a linux-based chain: DVD > handbrake+ffmpeg > mongoose 2.10 > roksbox > rokuXR > HDMI > 1920x1080 screen. I'm a newbie so any detail omissions that might be helpful, please speak up.

The NTSC "waking ned devine" main title (25 chapters) was encoded with handbrake 0.9.4 CLI, then post-processed with ffmpeg v0.6 straight copy. Unlike the +20 previous attempts, there was NO rebuffering at any of the test points (45, 49, 51, 59, 70 or 80 minutes). The roku's network refresh was the normal 'short-peak/long-valley' pattern and both initial and seek test loads were fast.

Update The below method was also successful on an August, 2001 release 4:3 KOYAANISQATSI dvd. Same relative changes in size, metadata and tbn. No rebuffers.

HandBrakeCLI -t 0 sees the source as

duration: 01:31:15; size: 720x480, pixel aspect: 8/9, display aspect: 1.33, 23.976 fps; autocrop: 100/106/8/6; English (AC3) (Dolby Surround) (iso639-2: eng), 48000Hz, 192000bps

Encoded from /dev/dvd with

--width 640 --vb 1800 --ab 192 --rate 23.976 --cfr --decomb --optimize --large-file --encoder ffmpeg --aencoder ac3

ffmpeg -i sees the mp4 output (rebuffers consistently at 47-49m) as

Seems stream 0 codec frame rate differs from container frame rate: 24000.00 (24000/1) -> 23.98 (24000/1001)
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'NEDDEVINE_1.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: mp42isom
    encoder         : HandBrake rev2965 2010060199
  Duration: 01:31:21.02, start: 0.000000, bitrate: 1538 kb/s
    Stream #0.0(und): Video: mpeg4, yuv420p, 640x272 [PAR 1:1 DAR 40:17], 1344 kb/s, 23.98 fps, 23.98 tbr, 90k tbn, 24k tbc
    Stream #0.1(eng): Audio: ac3, 48000 Hz, 2 channels, s16, 192 kb/s

2nd pass ffmpeg treatment (very fast)

ffmpeg -i NEDDEVINE_1.mp4 -acodec copy -vcodec copy copytest.mp4

Output is 180K larger with changes in metadata and tbn, but no change in duration.

Seems stream 0 codec frame rate differs from container frame rate: 24000.00 (24000/1) -> 23.98 (24000/1001)
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'copytest.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2mp41
    encoder         : Lavf52.64.2
  Duration: 01:31:21.02, start: 0.000000, bitrate: 1539 kb/s
    Stream #0.0(und): Video: mpeg4, yuv420p, 640x272 [PAR 1:1 DAR 40:17], 1344 kb/s, 23.98 fps, 23.98 tbr, 24k tbn, 24k tbc
    Stream #0.1(eng): Audio: ac3, 48000 Hz, 2 channels, s16, 192 kb/s