Roku Developer Program

Join our online forum to talk to Roku developers and fellow channel creators. Ask questions, share tips with the community, and find helpful resources.
cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
belltown
Roku Guru

Another JSON Parser [JSONDecoder]

I've written a new JSON parser. Here it is: https://github.com/belltown/Roku/blob/master/JSONDecoder/source/JSONDecoder.brs

There are already a couple of other JSON parsers out there: one by hoffmcs, http://forums.roku.com/viewtopic.php?f=34&t=30688&p=186934#p186934, and rdJSONParser in librokudev, see http://forums.roku.com/viewtopic.php?f=34&t=32208&p=200787 and https://github.com/rokudev/librokudev/blob/master/source/librokudev/source/rokudev_src/rdJSON.brs.

The hoffmcs parser is very slow, although does a fairly good job of parsing most JSON, although strings containing any UNICODE characters are treated as invalid. The librokudev parser is faster than the hoffmcs parser for smaller files, but still rather slow for larger files, and does not parse all valid JSON; for example, it fails to parse JSON text containing names that have characters such as "$", "." and "-", and it won't parse backslash-escaped characters in strings (except for the " character), and UNICODE characters are not parsed.

The new JSON parser should parse any valid JSON and is faster than the other parsers for larger JSON responses. It will also convert UNICODE characters that have ASCII equivalents.

The parser has no external dependencies. Simply copy the JSONDecoder.brs file into your source directory and use as follows:


decoderObject = JSONDecoder ()
json = decoderObject.decodeJSON ("<json formatted string>")
if json <> Invalid
' Handle the json object/array ...
else
print decoderObject.errorString ()
endif


A single JSONDecoder object may be used for multiple calls to decodeJSON ().
Unicode characters less than \u0080 (128) are converted to ASCII.
Unicode characters of \u0080 (128) and higher are converted to "." characters.
Characters with ASCII values of 128 and higher are converted to "." characters.
The JSON 'null' value is converted to 'invalid'.
Numbers are converted to integers unless they are too large, or contain a fraction or exponent, in which case a float is used.
For performance reasons, all valid JSON strings should be handled correctly, but not all invalid JSON will be detected.
If the input string is not valid JSON, the code should not crash, although it is not defined what will be returned.
Rudimentary error diagnostics are provided by calling errorString () if decodeJSON () returns Invalid.

I ran some tests on my Roku 1 to compare execution times for the different parsers using the Twitter API, changing the value of rpp to return different numbers of results:
http://search.twitter.com/search.json?q=roku&rpp=1&include_entities=true&result_type=mixed

rpp=1 => 2,304 bytes. JSONDecoder: 0.3 secs, rdJSONParser: 0.05 secs, hoffmcs parser: 1.0 secs
rpp=10 => 11,331 bytes. JSONDecoder: 1.7 secs, rdJSONParser: 0.4 secs, hoffmcs parser: 7.1 secs
rpp=50 => 47,000 bytes. JSONDecoder: 7.5 secs, rdJSONParser: 4.9 secs, hoffmcs parser: 57.8 secs
rpp=78 => 80,002 bytes. JSONDecoder: 12.5 secs, rdJSONParser: 12.6 secs, hoffmcs parser: 140.0 secs
rpp=100 => 101,195 bytes. JSONDecoder: 15.9 secs, rdJSONParser: 19.8 secs, hoffmcs parser: 212.3 secs

Obviously, XML responses are parsed more efficiently, since Roku has native support for XML. However, not all APIs provide XML output. For smaller JSON queries (less than 75K bytes), rdJSONParser is faster although may not necessarily parse all JSON responses. For larger queries, JSONDecoder seems to work faster and should parse all valid JSON responses. The hoffmcs parser seems slower in all cases.

Send a PM to belltown if you find any problems.
0 Kudos
7 REPLIES 7
Trevor
Visitor

Re: Another JSON Parser [JSONDecoder]

Very nice, thanks for sharing!
*** Trevor Anderson - bloggingwordpress.com - moviemavericks.com ***
0 Kudos
kbenson
Visitor

Re: Another JSON Parser [JSONDecoder]

"belltown" wrote:
I ran some tests on my Roku 1 to compare execution times for the different parsers using the Twitter API, changing the value of rpp to return different numbers of results:
http://search.twitter.com/search.json?q=roku&rpp=1&include_entities=true&result_type=mixed

rpp=1 => 2,304 bytes. JSONDecoder: 0.3 secs, rdJSONParser: 0.05 secs, hoffmcs parser: 1.0 secs
rpp=10 => 11,331 bytes. JSONDecoder: 1.7 secs, rdJSONParser: 0.4 secs, hoffmcs parser: 7.1 secs
rpp=50 => 47,000 bytes. JSONDecoder: 7.5 secs, rdJSONParser: 4.9 secs, hoffmcs parser: 57.8 secs
rpp=78 => 80,002 bytes. JSONDecoder: 12.5 secs, rdJSONParser: 12.6 secs, hoffmcs parser: 140.0 secs
rpp=100 => 101,195 bytes. JSONDecoder: 15.9 secs, rdJSONParser: 19.8 secs, hoffmcs parser: 212.3 secs

Obviously, XML responses are parsed more efficiently, since Roku has native support for XML. However, not all APIs provide XML output. For smaller JSON queries (less than 75K bytes), rdJSONParser is faster although may not necessarily parse all JSON responses. For larger queries, JSONDecoder seems to work faster and should parse all valid JSON responses. The hoffmcs parser seems slower in all cases.


Wow, great!

Considering how it's more correct in data it can parse and that it's almost as fast for most small inputs (and faster for large ones), I'd be happy to swap rdJSONParser out for your implementation if you're amenable. The whole purpose of librokudev is to be a community library of commonly needed routines, but that only works if people contribute. 🙂
-- GandK Labs
Check out Reversi! in the channel store!
0 Kudos
belltown
Roku Guru

Re: Another JSON Parser [JSONDecoder]

"kbenson" wrote:
Wow, great!

Considering how it's more correct in data it can parse and that it's almost as fast for most small inputs (and faster for large ones), I'd be happy to swap rdJSONParser out for your implementation if you're amenable. The whole purpose of librokudev is to be a community library of commonly needed routines, but that only works if people contribute. 🙂


You're welcome to add my code to librokudev, and make any changes to it that you see fit. Let me know if there's anything I need to do to make that happen. Since rdJSONParser has a different interface from my code and is faster for smaller inputs, even though it doesn't handle all JSON data, you might want to keep both versions around just in case someone has a preference for using one parser rather than the other.
0 Kudos
kbenson
Visitor

Re: Another JSON Parser [JSONDecoder]

"belltown" wrote:

You're welcome to add my code to librokudev, and make any changes to it that you see fit. Let me know if there's anything I need to do to make that happen. Since rdJSONParser has a different interface from my code and is faster for smaller inputs, even though it doesn't handle all JSON data, you might want to keep both versions around just in case someone has a preference for using one parser rather than the other.


Yeah, it will probably live on in some manner for backwards compatibility, but the run-time difference is small enough on most inputs that I think most people that would want to use it would be much happier with something less likely to blow up from an odd character.

Thanks!
-- GandK Labs
Check out Reversi! in the channel store!
0 Kudos
agmark
Visitor

Re: Another JSON Parser [JSONDecoder]

Thank you for your work on this belltown. JSON is used a lot and the Roku box needs all the help it can get in dealing with it. Maybe someday we'll get a native json component, but until then... :roll:
0 Kudos
gonzotek
Visitor

Re: Another JSON Parser [JSONDecoder]

Roku firmware 4.8 (and beyond) now includes a JSON parser: http://sdkdocs.roku.com/display/RokuSDK ... ctSincev48

It does not have support for encoding of Brightscript objects into json strings, just parsing from json strings to brightscript objects, so the json builder function found in libRokuDev may still be of utility to some people.
Remoku.tv - A free web app for Roku Remote Control!
Want to control your Roku from nearly any phone, computer or tablet? Get started at http://help.remoku.tv
by Apps4TV - Applications for television and beyond: http://www.apps4tv.com
0 Kudos
TheEndless
Channel Surfer

Re: Another JSON Parser [JSONDecoder]

"gonzotek" wrote:
Roku firmware 4.8 (and beyond) now includes a JSON parser: http://sdkdocs.roku.com/display/RokuSDK ... ctSincev48

It does not have support for encoding of Brightscript objects into json strings, just parsing from json strings to brightscript objects, so the json builder function found in libRokuDev may still be of utility to some people.

Probably worth noting that it's also only available in 4.8+, so you'd still need another library for JSON support on the Roku 1s (and logic to prevent a syntax error/crash on 3.x).
My Channels: http://roku.permanence.com - Twitter: @TheEndlessDev
Instant Watch Browser (NetflixIWB), Aquarium Screensaver (AQUARIUM), Clever Clocks Screensaver (CLEVERCLOCKS), iTunes Podcasts (ITPC), My Channels (MYCHANNELS)
0 Kudos