Roku Developer Program

[EnTerr]

"RokuMarkn" wrote:
The video player has a fixed size buffer, about 50-60 MB depending on the firmware and model. It keeps the buffer as full as possible to avoid rebuffering during brief network outages or slowdowns.

I've mentioned this previously, but I want to reiterate...

So over-buffering is the reason for page thrashing, degradation and consequently reboot?
Is the same reason behind the famed 2XS reboots they constantly bitch about in General forum?
As well as the fits Netflix SuperHD was having, it all seems to fit together.

The paragraph on VM - very informative, worth including in the docs so you won't have to re-iterate it in the future. Silly me, i did not think about that - was only thinking at BSc level. Have memories of the g(l)ory days of Applesoft Basic where had all of 48KB chock-full of data with only ~300 bytes free and quite often it will freeze for a minute, GC trying to re-arrange what's already packed.

Is there nothing that can be done? Like simply decrease the max buffering from 50 to 20mb - either globally or depending on model. Or if anybody has the time, box can auto-tune by using vmstat or somesuch. Or set traffic shaping on the network interfaces... pre-buffering 10 minutes of video is overkill since it comes at expense of stability

[RokuMarkn]

It's a matter of perspective -- to a developer trying to keep megabytes of data in memory while playing a video, it seems we're overbuffering. To a user who keeps getting rebuffers, it seems like we're underbuffering. Adjusting the buffer dynamically based on need is possible, but to the developer that would just mean that the same low memory problems would occur, but less frequently to fewer users and be even harder to debug.

Note that it must have been a very low bandwidth stream to play from the buffer for 10 minutes. 60 MB divided by 10 minutes is 800 kbps. At a more normal bitrate, say 3 mbps, the buffer only holds about 2.5 minutes of data.

--Mark

[destruk]

So how is it that people are always bragging about their system running for decades without a reboot on Linux? By your definition of thrashing and discarding pages slowing down, it can't be like that on all systems. Maybe it'd be better to only have 6 MB of memory in Roku. That way with less ram the entire system would be blazingly fast pretty much all the time.
https://www.novell.com/communities/cool ... n-you-top/

It sounds like you're saying roku buffers all the available memory (for the buffer aka 60mb) and then fragments the ram the longer you're using it which causes these crashes. There are a variety of solutions for your memory fragmentation issue from splitting the sandbox ram area and tmp storage for full wipe and rewrite operations to expanding the functionality of the microSD card - just wondering what items are of utmost importance that roku has been working on which prevent them from fixing basic "memory leaks" and issues like these for stability.

It would be great if this issue was fixed, so if I was watching a channel and continuously played short 2-minute videos I wouldn't crash the device. I realize fixng real problems like these isn't as fun and enjoyable as say adding a new roListScreen component or supporting more 3d components - but which is really more important? Having a stable/usable core system to do what you need to do, or adding more bells and whistles to a broken foundation?

[RokuMarkn]

Systems with a disk (writable mass storage) are a very different situation. Then ANY page, not just the ones that are copies of read only data, can be discarded and reused (perhaps after writing it to the swap file first). Available (virtual) memory on such systems is limited only by the size of the swap file, which is usually much larger than physical memory. The other difference is in non-embedded systems, if a process uses too much memory, it can be killed. In the Roku system, there is effectively only one process, and killing it looks the same as crashing the system to the user.

I actually have no idea if the OP's original problem in this thread has anything to do with memory usage. I'd bet against it. I also have no idea if the problem you've just mentioned (continuously playing short videos) has anything to do with the OP's problem -- I'd also bet against that since that's a completely different use case.

--Mark

[destruk]

So what would you bet the issue is if not bandwidth (as stated), or memory fragmentation? Maybe it's overheating because the bits are flying through the controller too fast or what?

[greubel]

Let me restate the problem.
1. Movie is mp4 and has a size of 875.3 MB on disc. 720x390 h.264, AAC duration is 1:49:35
2. Wifi rate 12 - 24 mbps, Lan rate 40+ mbps. This is from Roku's bandwidth log entry.
3. Movie will play between 3 and 1:40 minutes and random points in between before Roku crashes.
4. Played the movie from the USB and it plays ok.

True: my channel is maxed out at the 3x package limit and I use a lot of memory and a lot of tmp:
I can force my media server to clamp the transfer rate (below 10 mbps) and the movie will play.
But I can't do that for other media servers (Twonky,Serviio) or internet transfers.

Roku(3.1 1198) Model N1000
Roku(3.1 1198) Model 2100X
Roku(5.5 453) Model 3100X <-------------- fails
Roku(5.5 415) Model 4200X

I have reduced my memory usage and eliminated any unnecessary network processing.

[EnTerr]

"RokuMarkn" wrote:
I actually have no idea if the OP's original problem in this thread has anything to do with memory usage.

Hm, we are in a sea of trouble if even high priest cannot tell if the issue is memory related.

Is there no way to make runGarbageCollector() report at least how much memory is used total ?
And/or to check how much of the tmpfs is used and how much free ? (roFileSystem.getVolumeInfo() seems a useful place and we are talking about all of tmp here, not just package's own tmp:/ dir)

Having such numbers will allow developers to diagnose memo issues and maybe even make channels self-tune when they smell danger.

[greubel]

I updated my dump() routine to include file system usage.

If memory thrashing is the problem, should we stop ALL background activity while the video is playing ?
That might prevent page faults !

Memory Allocated 321,293 bytes - total items 2,867
*
Disk storage used = 25,549,792
*
Is this out of range ? Too big ?
*
These are the numbers of allocated objects .
Memory Items

roArray =  225
roAudioPlayer =  1
roImageCanvas =  4
roSystemLog =  1
roAudioResource =  3
roDatagramSocket =  6
roSocketEvent =  4
roMessagePort =  3
roBoolean =  93
String =  863
roInvalid =  15
roInt =  6
roVideoScreenEvent =  1
roAssociativeArray =  336
roDeviceInfo =  1
roVideoScreen =  1
Integer =  6
roTimespan =  1
roInteger =  643
roString =  599
roInput =  1
roFilesystem =  1
roFontRegistry =  1
roFunction =  17
roByteArray =  15
roStreamSocket =  3
Invalid =  17

File System with sizes

* common:/
   LibCore/
      v30/
         bslCore.brs   8030
         bslDefender.brs   8660
   certs/
      ca-bundle.crt   230732
* pkg:/
   manifest   568
   media/
      images/
         Chaneru_Center_HD.jpg   22259
         Chaneru_Center_SD.jpg   11422
         Chaneru_Side_HD.jpg   2969
         Chaneru_Side_SD.jpg   3030
         Main_BG.jpg   14946
         OffSlice.jpg   694
         OnSlice.jpg   725
         PreSlice.jpg   737
         Slice_Default.jpg   3127
         TopSlice.jpg   733
         USB_Off.png   7463
         USB_On.png   10621
         audio.png   11476
         black.jpg   940
         certified.png   7994
         control_slice.jpg   926
         delete.png   11079
         down.png   7372
         folder.png   9435
         folders.png   14242
         fresh.png   6777
         help.png   9906
         infinity.png   10973
         logo.png   16835
         loop.png   10441
         main_slice.jpg   4255
         media.png   8312
         next.png   10355
         off.png   3353
         on.png   3355
         once.png   9820
         option.png   11090
         pause.png   9962
         photo.png   21577
         play.png   10007
         player.png   10397
         playlist.png   4868
         pls.png   9455
         popcorn.png   5940
         previous.png   10271
         qmark.png   18723
         r_back.png   4825
         r_ff.png   4473
         r_home.png   5442
         r_info.png   5886
         r_pause.png   4730
         r_replay.png   6155
         r_rew.png   3716
         red.jpg   649
         refresh.png   11086
         repeat.png   11382
         rotten.png   6949
         rottentomato.png   11007
         search.png   10299
         shuffle.png   10730
         sm_logo.png   18695
         spilled.png   5902
         splash_logo_HD.jpg   15660
         splash_logo_SD.jpg   6902
         transparent.png   2885
         up.png   7281
         video.png   20824
         vlc.png   9573
         wma.png   8438
      sounds/
         scroll.wav   20306
         select.wav   41186
         serror.wav   29942
   source/
      Access.brs   9700
      Audio.brs   14058
      Chaneru.brs   4018
      Connect.brs   20684
      Control.brs   13118
      Database.brs   43492
      Device.brs   9343
      Directory.brs   28806
      Errno.brs   7619
      Font.brs   15131
      Google.brs   6977
      Help.brs   1229
      History.brs   14673
      Info.brs   8165
      Keyin.brs   938
      List.brs   13319
      Logger.brs   5801
      Mac.brs   5136
      Main.brs   10702
      MsgPort.brs   13926
      NewClean.brs   9640
      OldClean.brs   10128
      Option.brs   20516
      Photos.brs   6490
      Playlist.brs   20344
      Remote.brs   1948
      Roku.dev   5462
      Roku.layout   3351
      Screen.brs   4477
      Search.brs   3910
      Soap.brs   9191
      Speed.brs   8865
      Stream.brs   13284
      Subs.brs   82985
      Subscribe.brs   8992
      Test.brs   66403
      Theme.brs   19653
      Theme.xml   4630
      UDP.brs   22954
      USB.brs   3465
      VLC_options.brs   6572
      Video.brs   12496
      _Change Log.txt   2416
      iLoader.brs   7161
      iPhoto.brs   9257
* tmp:/
   Theme/
      Chaneru2/
         Arial Unicode.ttf   23278008
         background.jpg   222575
      USB/
         help.png   6604
         option.png   9035
         player.png   7594
         shuffle.png   8748
         play.png   7728
         refresh.png   9259
         loop.png   9246
         delete.png   10723
         stop.png   6457
         next.png   7269
         once.png   8633
         pause.png   7014
         infinity.png   11011
         previous.png   7328
         repeat.png   11523
         TopSlice.jpg   733
         folder.png   7645
         folders.png   8655
         photo.png   16380
         audio.png   11476
         playlist.png   4868
         USB_Off.png   7179
         vlc.png   9573
         down.png   7372
         USB_On.png   9359
         off.png   3353
         video.png   13880
         up.png   7281
         OffSlice.jpg   694
         OnSlice.jpg   725
         Main_BG.jpg   14946
      Default/
         help.png   9906
         option.png   11090
         player.png   10397
         shuffle.png   10730
         play.png   10007
         refresh.png   11086
         loop.png   10441
         delete.png   11079
         next.png   10355
         once.png   9820
         pause.png   9962
         infinity.png   10973
         previous.png   10271
         repeat.png   11382
         search.png   10299
         PreSlice.jpg   737
         TopSlice.jpg   733
         OffSlice.jpg   694
         OnSlice.jpg   725
         folder.png   9435
         folders.png   14242
         photo.png   21577
         audio.png   11476
         playlist.png   4868
         USB_Off.png   7463
         vlc.png   9573
         down.png   7372
         USB_On.png   10621
         off.png   3353
         video.png   20824
         up.png   7281
         Main_BG.jpg   14946

[TheEndless]

"greubel" wrote:
Disk storage used = 25,549,792
*
Is this out of range ? Too big ?

In my personal experience, I'd say yes to this. I've had instability issues storing more then ~10 megs in tmp:, so I have a LRU routine that keeps it below that.

[greubel]

This dump had a font stored in tmp: Arial Unicode.ttf 23,278,008
So without the font, which I have tried, it still fails. That would knock tmp down to 2.3 meg.