Forum Discussion

destruk's avatar
destruk
Streaming Star
9 years ago

Parsing JSON v XML

Parsing the content as JSON appears to be twice as fast as parsing XML.  It is a big change to the server code to switch existing feeds over to json, but the payoff is worth it.  I'll need to run some profiling now to determine optimization which will require two sets of server code to compare with two versions of channel code.

[spun off viewtopic.php?f=34&t=101573 --RokuNB]
Roku Developer Program

28 Replies

Show More
  • destruk's avatar
    destruk
    Streaming Star
    9 years ago
    As far as the render thread being a hot potato, if nothing on the screen requires an update what real difference does it make if it has nothing to draw while it waits for the task to finish with a "BUSY" text string on the screen?  It sounds like a whole lot of wasted processing time being used updating the screen with a static image that is already there 60 times/second.
  • NB_'s avatar
    NB_
    Roku Guru
    9 years ago
    "destruk" wrote:
    I think the main question I have that would be useful to answer for everybody is - json and xml feeds are downloaded using rourltransfer inside a task node.  The task thread does that just fine.
    Now once it downloads the data completely we throw the result to the "content" interface which is specified as a string type for the field in the task node xml which makes it accessible to the main thread.  I've been doing it this way because of the note on thread rendevous between the main thread and the task thread being super slow, so by grabbing the text string in the main thread and parsing it there and then assigning or building the content to populate the screen with is much faster.

    What "note on thread rendezvous [...] being super slow"? Please point a source - is this in the documentation or forum folklore?

    I did try running the parse inside the task thread and that part appears to be just as fast as running a single command for parse in the main thread.
    However, if I want the task node itself to assign and build the content nodes once the string has been parsed for the data it needs to do that, how can we move, copy, reference, or otherwise get that data to the main render thread to populate the screen?  Nodes created inside a task node are owned and controlled by that task node, we can't change the content field of the task node to be an object type to make it available to the main render thread?  How else would we do it besides sending the string to the main thread to do that part?

    Please don't say "main" anywhere near "render" thread - it is confusing (me, you, everyone) ;). The render thread has one job - painting the UI, that does not make it main. You can pass simple types, arrays, dictionaries, nodes just fine between a task and render thread. If i remember there were some rules which thread "owns" nodes (something like visual nodes are always owned by the render thread, where Node and ContentNode belong to the creator but can change ownership on assignment or such) - that does not mean another thread cannot access them though, just that rendezvous would be used under the hood
  • destruk's avatar
    destruk
    Streaming Star
    9 years ago
    You should avoid as many rendezvous operations as possible to ensure maximum performance of your application. It is better to build an entire tree of nodes or ContentNode nodes, then pass the tree to the render thread using one rendezvous, than to repeatedly pass each node in the tree as it is created. For field setting and getting, ifSGNodeField[/url:30cawjny] methods such as 
    getFields()

     and 
    setFields()

    , which set and get multiple fields at once, should be used rather than several get and set operations.
  • destruk's avatar
    destruk
    Streaming Star
    9 years ago
    In a task node thread, parsing and building a contentnode tree with 320 children takes minutes.  In the render thread doing that takes seconds, therefor, real world tests indicate that the taskl node thread is super slow.  And then when the content node is done building it, how does the thread with the program execution get the content back to work with?  I did have a bug with my search code where it returned 4889 individual content items, so I tightened the search criteria to get a more reasonable number of content items returned (80) - but if it had to build close to 5000 content nodes at one time, even in a task node that would appear to take hours.

    If I set the task node content field to type "node" it didn't appear to make the contentnode tree it built accessible to anything outside the task node.  Is this a bug, or did I just miss something important I had to do to access it?
  • NB_'s avatar
    NB_
    Roku Guru
    9 years ago
    "destruk" wrote:
    You should avoid as many rendezvous operations as possible to ensure maximum performance of your application. It is better to build an entire tree of nodes or ContentNode nodes, then pass the tree to the render thread using one rendezvous, than to repeatedly pass each node in the tree as it is created.

    Nobody questions that part - but what makes you think that doing the parsing not in "task" but in "main" (not render!) thread would make any difference? It won't, as far as i know they have the same priority. Besides, it makes no difference when you have to hand the created object to the render thread, whether it comes from main or task...
  • destruk's avatar
    destruk
    Streaming Star
    9 years ago
    Actually - if the Task node doesn't make the content node available to anything else, then you can't do it this way, can you?  Or are you saying the task should create the content list, and then the task should find the grid or screen and tell it to populate itself using the task node's content it made, from within the task itself?  That way might work - it doesn't involve interface fields for the content.
    But if the interface field is set to node, or object, is it possible to get that interface data from somewhere else?  That's the flip side.
  • NB_'s avatar
    NB_
    Roku Guru
    9 years ago
    "destruk" wrote:
    In a task node thread, parsing and building a contentnode tree with 320 children takes minutes.  In the render thread doing that takes seconds, therefor, real world tests indicate that the taskl node thread is super slow.

    It's probably the 3rd time i ask this: how do you "parse" XML/JSON in the render thread, given parseXml()/parseJSON() are not available in a render thread? - please reply!

    And then when the content node is done building it, how does the thread with the program execution get the content back to work with?  [...] If I set the task node content field to type "node" it didn't appear to make the contentnode tree it built accessible to anything outside the task node.  Is this a bug, or did I just miss something important I had to do to access it?

    You just assign or access the Node to the other component, that's it - and it should work. You must be missing something.
  • destruk's avatar
    destruk
    Streaming Star
    9 years ago
    RokuNB -
    In the channels main.brs, I create a roSGScreen
    Next I use CreateScene for my main scenegraph thread.  This is what I think of when I say "Main thread, or render thread" As it is the first screen the app displays.
    If that isn't the render thread then I don't know what the render thread would be?
    Parse and ParseJSON both work in that thread - the first one created for the scene.
    They timeout if it takes more than 3 seconds in that thread to do the work to build content nodes from the string returned from the task thread with rourltransfer.
    It says "execution timeout" when this happens.
  • destruk's avatar
    destruk
    Streaming Star
    9 years ago
    As for the task node itself, I use m.downloaddata=m.top.findnode("DownloadData")
    I can set the control to "start" and it downloads, then it fills the "content" interface with the downloaded string, and the main scene builds the contentnodes to populate the screen with.
    If you are getting the information from m.downloaddata.content to do this, then it is building the nodes in the thread where that line appears.

    If however, the content node is using setfield and/or setfields within itself based on the data it downloaded within itself to create the content nodes, then how do you get that data to where you can use it?  Do you say, have a component called rowlist, and you have it m.rowlist.setcontent(m.downloaddata.content) after populating the m.downloaddata.content interface with the contentnode and set to type "node" ?

    I can try some more things here as I just changed all the feeds to json and optimized the server for fields that aren't used, etc etc.  Thanks again for your help.