Roku Developer Program

[Romans_I_XVI]

So first of all I'd like to say I'd absolutely love it if we had an option to keep an audio file loaded in memory, since the main issue with looping is that when the file is played again it is loaded again which takes a few seconds, also by default roAudioPlayer assumes the player is going to be fetching the audio file from the specified "url" which is probably the core of the problem, it's designed to play music off of a remote server.

That said, that's not the point of this post. The point is to explore which audio format is best for the smoothest possible looping. There was a time that, surprisingly enough, WMA was the best format for looping. It took the least amount of time to start so the gap between the end and start of a track was acceptable. That said there is currently what I would consider a bug with WMA playback on newer devices, songs always start a second or so in. It's as if playback starts, but the file is still loading so the user doesn't actually hear anything, and then when it's loaded it's about a second in to the song. This sounds worse than having a gap, so I will probably be moving away from WMA.

I've been testing all the officially supported file formats. That is: AAC, MP3, WMA, WAV (PCM), AIFF, FLAC, ALAC, AC3, E-AC3 . Here's what I've come up with so far, just based on what I hear when using the different formats.

• AIFF, FLAC, ALAC, and WAV - These obviously produce the larger files since they are lossless audio formats and WAV of course is not compressed at all. Based on that they seem to take the longest to playback. Again I attribute that to the way roAudioPlayer treats audio playback, how it seems to assume audio files will be loading off of a server even if they are stored locally.


• MP3, AC3, E-AC3 - These still have a decent gap in their playback. There might be minor differences between them but the gap was enough that I wasn't interested in digging in to them individually.


• WMA - Still has a decent playback startup time, but as mentioned seems to have a bug on newer Roku models (I've tested on Roku Ultra, Roku Stick (3800X), and TCL Roku TV)


• AAC - THE WINNER! This is the very clear and obvious winner in terms of playback startup time. There is still a gap but it is small enough that I believe I can work with it.

A M4A file with the AAC format seems to give the best results and after digging in to the roAudioPlayerEvent messages I think I know why. I'm not even sure where the documentation is on this, but I set up the roAudioPlayerEvent to spit out anything from GetMessage() and GetInfo() and this is the series of events that fire during playback of an AAC format file.

GetMessage() :: startup progress
GetInfo() :: invalid
GetMessage() :: startup progress
GetInfo() :: invalid
GetMessage() :: startup progress
GetInfo() :: invalid
GetMessage() :: startup progress
GetInfo() :: invalid
GetMessage() :: 
GetInfo() :: invalid
isListItemSelected().GetIndex() :  0
GetMessage() :: startup progress
GetInfo() :: invalid
GetMessage() :: startup progress
GetInfo() :: invalid
GetMessage() :: Segment download started
GetInfo() :: <Component: roAssociativeArray> =
{
    EndTime: 0
    SegBitrate: 0
    Sequence: 1
    StartTime: 0
}
GetMessage() :: startup progress
GetInfo() :: invalid
GetMessage() :: startup progress
GetInfo() :: invalid
GetMessage() :: startup progress
GetInfo() :: invalid
GetMessage() :: startup progress
GetInfo() :: invalid
GetMessage() :: startup progress
GetInfo() :: invalid
GetMessage() :: startup progress
GetInfo() :: invalid
GetMessage() :: Stream started.
GetInfo() :: <Component: roAssociativeArray> =
{
    IsUnderrun: false
    MeasuredBitrate: 9765
    StreamBitrate: 128000
    Url: ""
}
GetMessage() :: Format Detected
GetInfo() :: <Component: roAssociativeArray> =
{
    audio: "AAC"
    captions: "NONE"
    video: "NONE"
}
GetMessage() :: Download segment info
GetInfo() :: <Component: roAssociativeArray> =
{
    Bitrate: 0
    BufferLevel: 0
    BufferSize: 0
    DownloadDuration: 23
    IPAddress: ""
    SegBitrate: 0
    SegSize: 1047756
    SegType: 0
    SegUrl: ""
    Sequence: 1
    Status: 0
}
GetMessage() :: startup progress
GetInfo() :: invalid
GetMessage() :: Segment download started
GetInfo() :: <Component: roAssociativeArray> =
{
    EndTime: 0
    SegBitrate: 0
    Sequence: 2
    StartTime: 0
}
GetMessage() :: Download segment info
GetInfo() :: <Component: roAssociativeArray> =
{
    Bitrate: 0
    BufferLevel: 0
    BufferSize: 0
    DownloadDuration: 18
    IPAddress: ""
    SegBitrate: 0
    SegSize: 1048560
    SegType: 0
    SegUrl: ""
    Sequence: 2
    Status: 0
}
GetMessage() :: Segment download started
GetInfo() :: <Component: roAssociativeArray> =
{
    EndTime: 0
    SegBitrate: 0
    Sequence: 3
    StartTime: 0
}
GetMessage() :: Download segment info
GetInfo() :: <Component: roAssociativeArray> =
{
    Bitrate: 0
    BufferLevel: 0
    BufferSize: 0
    DownloadDuration: 17
    IPAddress: ""
    SegBitrate: 0
    SegSize: 1047966
    SegType: 0
    SegUrl: ""
    Sequence: 3
    Status: 0
}
GetMessage() :: Segment download started
GetInfo() :: <Component: roAssociativeArray> =
{
    EndTime: 0
    SegBitrate: 0
    Sequence: 4
    StartTime: 0
}
GetMessage() :: Download segment info
GetInfo() :: <Component: roAssociativeArray> =
{
    Bitrate: 0
    BufferLevel: 0
    BufferSize: 0
    DownloadDuration: 17
    IPAddress: ""
    SegBitrate: 0
    SegSize: 1048242
    SegType: 0
    SegUrl: ""
    Sequence: 4
    Status: 0
}
GetMessage() :: Segment download started
GetInfo() :: <Component: roAssociativeArray> =
{
    EndTime: 0
    SegBitrate: 0
    Sequence: 5
    StartTime: 0
}
GetMessage() :: Download segment info
GetInfo() :: <Component: roAssociativeArray> =
{
    Bitrate: 0
    BufferLevel: 0
    BufferSize: 0
    DownloadDuration: 7
    IPAddress: ""
    SegBitrate: 0
    SegSize: 533439
    SegType: 0
    SegUrl: ""
    Sequence: 5
    Status: 0
}
GetMessage() :: startup progress
GetInfo() :: invalid
GetMessage() :: startup progress
GetInfo() :: invalid
GetMessage() :: startup progress
GetInfo() :: invalid
GetMessage() :: start of play
GetInfo() :: invalid
GetMessage() :: Stream segment
GetInfo() :: <Component: roAssociativeArray> =
{
    SegStartTime: 0
    SegUrl: ""
    Sequence: 1
    StreamBandwidth: 0
}
GetMessage() :: Stream segment
GetInfo() :: <Component: roAssociativeArray> =
{
    SegStartTime: 28885
    SegUrl: ""
    Sequence: 2
    StreamBandwidth: 0
}
GetMessage() :: Stream segment
GetInfo() :: <Component: roAssociativeArray> =
{
    SegStartTime: 55426
    SegUrl: ""
    Sequence: 3
    StreamBandwidth: 0
}
GetMessage() :: Stream segment
GetInfo() :: <Component: roAssociativeArray> =
{
    SegStartTime: 80039
    SegUrl: ""
    Sequence: 4
    StreamBandwidth: 0
}
GetMessage() :: 
GetInfo() :: invalid
GetMessage() :: end of stream
GetInfo() :: invalid
GetMessage() :: Playback completed.
GetInfo() :: invalid
GetMessage() :: end of playlist
GetInfo() :: invalid

It seems with this format the Roku is "streaming" the file in segments. I imagine if it doesn't need to load the whole file before playback and instead starts with the first segment it loads faster. It's nice to have discovered, since WMA seems messed up right now I'll probably switch to using this. The whole thing still seems a bit silly to me though. It would be really nice if locally stored files weren't treated the same as a file that's streaming off of a server.

I don't personally know much about audio in general. If anyone else has any insights in to this I would be very interested. Also if anyone else has any ideas of potentially loading an audio file even faster I would very very much be interested. All of my games use locally stored looping music to some degree.

[destruk]

If you use a shoutcast server then the looping is done on the server for you, flawlessly.  Even Winamp has a shoutcast plugin.

[Komag]

I haven't yet tackled fully the game music system I will build for my game, but my brainstorm was to use a roAudioResource clip to cover the gap, and try to crossfade the music with the short sound. Have you already tried that?

[Romans_I_XVI]

Well roAudioResource really doesn't work well either. On most devices you can only play one sound at a time, and I am using it for gameplay sounds already.

[Komag]

It used to be one at a time, but now they all can do two at a time. But yeah, need to be judicious with it as it's needed for sound effects.

[necrotek]

It would be great if we had an audio Buffer we could have access to for sound mixing and playback.   I tried writing a mod/xm player in brightscript, but since I need to write each .wav file to the device which takes over a second and the fact that brightscript can be slow at parsing large arrays. That didn't work out so well in real time. I know that we are using a scripting language on top of a low powered device but this is something that an Amiga could do with a 7 mhz processor and 512k memory with plenty of cycles left to run a game.  

[Komag]

Yeah, just another example of Roku decidedly NOT being a gaming machine. But we work with what we've got, right?

[Romans_I_XVI]

Yes, it's very obvious that the audio player was designed for a specific purpose, playing a playlist of songs off of the internet. But yeah, we work with what we got. I'm just thankful that Roku had gaming in it's sights at one point in it's history, because if they didn't we wouldn't have some of the gaming specific functionality such as collision detection. Trust me, I've tried doing my own collision detection in BrightScript and it is sloooow by comparison. So yeah, we work with what we've got and be thankful for what we do have.

[marcelo_cabral]

I second that request, to have the player identify that the file is local and keep it on a cache would be great!

BTW thanks for information about AAC, I have the same issue with looping MP3 on my games, never had tried that!

[Komag]

wma files have stopped working for me in OS 10.5.0 (on a Roku Ultra 4800). I tested a couple other games, which work fine (RUN-CMD, Snake), so I assume those don't use wma (anymore?). I tested a game that hasn't been updated recently, Pathogen 2 (last update 2016), and the music there DOESN'T play for me, so I'm guess it's wma.

Has support for wma been removed entirely?

Also, I tried exporting audio from Audacity to m4a (AAC), and it worked, but the file size for the same bitrate was nearly double (compared to wma and mp3).

I assume Romans_I_XVI and other devs have updated your games with new audio files?