jbrave
Channel Surfer
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
03-31-2011
07:41 PM
dealing with large numbers of videos
In a grid screen, I'm retriving like 20 rows of 80-90 videos each. The parse time is about 1 second per row, but the http time is about 5 seconds per row. I thought about dynamically loading when the user moves close to the end of a row, or loading the next row when the user moves down (this would be pretty annoying to interrupt the user each time). I also wonder if I use asyncgettostring in a wait loop with say 100ms wait time, I could set up the data transfer and do a fake multitasking, where the user could do their thing and the data could continue loading. Assuming that even works however, how would one handle it if the user selects and plays a video - you wouldn't want to force them to reload all 2000 videos again - currently, I load the videos when the screen loads and store them into an array which I reload after the video finishes playing, this is very fast. I'm not sure how to keep track of what has been loaded and what has not, so the process can continue.
Probably I'll just put the data transfer up front when the channel loads, but if anyone has any suggestions, it would be very helpful.
- Joel
Probably I'll just put the data transfer up front when the channel loads, but if anyone has any suggestions, it would be very helpful.
- Joel
Screenshades: The first Screensaver for Roku2!
Musiclouds: The best free internet music, on your Roku!
Ouroborialis: Psychedelic Screensaver for Roku!
Musiclouds: The best free internet music, on your Roku!
Ouroborialis: Psychedelic Screensaver for Roku!
15 REPLIES 15
kbenson
Visitor
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
03-31-2011
09:09 PM
Re: dealing with large numbers of videos
BrightScript isn't the speediest interpreted language, but 1 second to parse a row of 80-90 items seems way too slow. What do you mean exactly when you say "parse"?
-- GandK Labs
Check out Reversi! in the channel store!
Check out Reversi! in the channel store!
jbrave
Channel Surfer
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-01-2011
06:29 AM
Re: dealing with large numbers of videos
I mean take the xml file, go
"if xml.parse(xmlxfer) then
for each video in xml.videos..."
and convert it to a set of posters. I was using timer.mark() and timer.totalseconds() to time each action, but it might round up to a second. Roku parses the XML fast enough as far as I'm concerned. Its actually getting the files that is an issue.
While I'm talking, here's a question:
If I use asyncgettostring() and a message port with a short wait time in my loop like msg=wait(25,port) can I do something like update the screen graphics while I'm waiting for my xml to come through, or is there a chance I'll miss the incoming data?
- Joel
"if xml.parse(xmlxfer) then
for each video in xml.videos..."
and convert it to a set of posters. I was using timer.mark() and timer.totalseconds() to time each action, but it might round up to a second. Roku parses the XML fast enough as far as I'm concerned. Its actually getting the files that is an issue.
While I'm talking, here's a question:
If I use asyncgettostring() and a message port with a short wait time in my loop like msg=wait(25,port) can I do something like update the screen graphics while I'm waiting for my xml to come through, or is there a chance I'll miss the incoming data?
- Joel
Screenshades: The first Screensaver for Roku2!
Musiclouds: The best free internet music, on your Roku!
Ouroborialis: Psychedelic Screensaver for Roku!
Musiclouds: The best free internet music, on your Roku!
Ouroborialis: Psychedelic Screensaver for Roku!

TheEndless
Channel Surfer
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-01-2011
11:10 AM
Re: dealing with large numbers of videos
If the most significant delay is during the web request, then doing an AsyncGetToString would allow you to continue processing messages/events. If the delay is more during the actual Parse(), then that won't help, since there's no way to asynchronously parse an XML string.
As for handling video selections during the AsyncGet, you can call AsyncCancel to kill the web request, and process the selection event. You'll just need to keep track of any rows that haven't completed loading, so you can continue with the load when the user returns to that screen.
Also, it probably goes without saying, but just to be safe, you want to make sure the port you're listening to is the same for both the screen and the roUrlTransfer object.
As for handling video selections during the AsyncGet, you can call AsyncCancel to kill the web request, and process the selection event. You'll just need to keep track of any rows that haven't completed loading, so you can continue with the load when the user returns to that screen.
Also, it probably goes without saying, but just to be safe, you want to make sure the port you're listening to is the same for both the screen and the roUrlTransfer object.
My Channels: http://roku.permanence.com - Twitter: @TheEndlessDev
Instant Watch Browser (NetflixIWB), Aquarium Screensaver (AQUARIUM), Clever Clocks Screensaver (CLEVERCLOCKS), iTunes Podcasts (ITPC), My Channels (MYCHANNELS)
Instant Watch Browser (NetflixIWB), Aquarium Screensaver (AQUARIUM), Clever Clocks Screensaver (CLEVERCLOCKS), iTunes Podcasts (ITPC), My Channels (MYCHANNELS)
destruk
Streaming Star
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-01-2011
12:02 PM
Re: dealing with large numbers of videos
Having it load 2000 videos, images, and data for a single selection screen sounds like too much to me. It could be a better idea having it load maybe 10 items per category with a "More" icon to load the rest in that category on a new screen. That would keep the traffic to a more reasonable level and not require you to load or cache so many poster images. Maybe include a dynamic search screen so users can get to that single film they want to see a lot quicker. How do you get past the >100 xml entries memory issue if you're loading 2000 films? If you're trying to stress test the box, your way is the way to do it, but I'd expect it to be slow, and clunky when it's being slammed with too much data to keep track of in it's tiny little brain. 🙂
As far as reloading data, I think you can tell it what item was selected with the row/column, and you don't need to close the gridscreen to play a video (either this current firmware or the testing firmware has that fix), so you shouldn't have to redraw the screen anymore after the first draw.
As far as reloading data, I think you can tell it what item was selected with the row/column, and you don't need to close the gridscreen to play a video (either this current firmware or the testing firmware has that fix), so you shouldn't have to redraw the screen anymore after the first draw.

TheEndless
Channel Surfer
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-01-2011
12:08 PM
Re: dealing with large numbers of videos
"destruk" wrote:
How do you get past the >100 xml entries memory issue if you're loading 2000 films?
What >100 xml entry memory issue? I've never experienced any limitation of that sort... at least not as low as 100.
And, for what it's worth, the Netflix channel loads 25+ rows of anywhere from 5 to 50 titles each on its main screen.
My Channels: http://roku.permanence.com - Twitter: @TheEndlessDev
Instant Watch Browser (NetflixIWB), Aquarium Screensaver (AQUARIUM), Clever Clocks Screensaver (CLEVERCLOCKS), iTunes Podcasts (ITPC), My Channels (MYCHANNELS)
Instant Watch Browser (NetflixIWB), Aquarium Screensaver (AQUARIUM), Clever Clocks Screensaver (CLEVERCLOCKS), iTunes Podcasts (ITPC), My Channels (MYCHANNELS)
kbenson
Visitor
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-01-2011
12:51 PM
Re: dealing with large numbers of videos
"jbrave" wrote:
I was using timer.mark() and timer.totalseconds() to time each action, but it might round up to a second.
timer.totalmilliseconds() is probably what you want.
-- GandK Labs
Check out Reversi! in the channel store!
Check out Reversi! in the channel store!
destruk
Streaming Star
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-01-2011
02:25 PM
Re: dealing with large numbers of videos
Does anyone here read threads or pay attention to what is posted anymore? geesh.
viewtopic.php?f=34&t=35105&p=226658&hilit=60+xml+100#p226658
ROKUKevin posted
"It sounds like your xml files are too large.... 100's of items will consume too much memory on the Roku.
It would be better to implement some sort of paging or add more hierarchies to your navigation structure.
--Kevin"
As to the size -
"There is no hard cap, but you will run into slow page loads with (retrieving...) messages....
--Kevin"
So I would think, that perhaps the reason it's slow is because you are having it load a lot and putting a strain on the memory limitations of the roku device? maybe?
--edited to be nicer, sorry-- Limiting your xml transfers to 60 or so programs keeps it quick for me, so that's what I do.
viewtopic.php?f=34&t=35105&p=226658&hilit=60+xml+100#p226658
ROKUKevin posted
"It sounds like your xml files are too large.... 100's of items will consume too much memory on the Roku.
It would be better to implement some sort of paging or add more hierarchies to your navigation structure.
--Kevin"
As to the size -
"There is no hard cap, but you will run into slow page loads with (retrieving...) messages....
--Kevin"
So I would think, that perhaps the reason it's slow is because you are having it load a lot and putting a strain on the memory limitations of the roku device? maybe?
--edited to be nicer, sorry-- Limiting your xml transfers to 60 or so programs keeps it quick for me, so that's what I do.

TheEndless
Channel Surfer
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-01-2011
04:32 PM
Re: dealing with large numbers of videos
"destruk" wrote:
Does anyone here read threads or pay attention to what is posted anymore? geesh.
viewtopic.php?f=34&t=35105&p=226658&hilit=60+xml+100#p226658
With all due respect, I think you're reading that too literally. The number of items in the XML is not the issue, but rather the size of the XML string being parsed. It also says "100's of items", not "100 items" and is specific to the memory consumed by the XML parser, not the data displayed on the current screen. That aside, jbrave is loading 20 different XML files with 80-90 items each, not a single XML file with 2000 items, so he is already implementing paged data as RokuKevin suggests in that thread.
My Channels: http://roku.permanence.com - Twitter: @TheEndlessDev
Instant Watch Browser (NetflixIWB), Aquarium Screensaver (AQUARIUM), Clever Clocks Screensaver (CLEVERCLOCKS), iTunes Podcasts (ITPC), My Channels (MYCHANNELS)
Instant Watch Browser (NetflixIWB), Aquarium Screensaver (AQUARIUM), Clever Clocks Screensaver (CLEVERCLOCKS), iTunes Podcasts (ITPC), My Channels (MYCHANNELS)
kbenson
Visitor
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-01-2011
06:08 PM
Re: dealing with large numbers of videos
For anyone who has a back-end that it's easily implemented in, JSON parsing is pretty quick, and results in a native BirghtScript associative array.
The link I presented is part of librokudev, which for anyone new might include a few functions you'll find helpful.
The link I presented is part of librokudev, which for anyone new might include a few functions you'll find helpful.
-- GandK Labs
Check out Reversi! in the channel store!
Check out Reversi! in the channel store!