Roku Developer Program

[ratish]

Hi

I am facing a unfimiliar issue using roUrlTransfer component to fetch XML which is UTF-16 encoded. The synchrounous and asynchrounous call to get the response as string and file returns successfully but content seems to be near empty. In my case when i tried to print the string returned by roUrlTransfer object's GetToString(), it shows only a opening tag("<"). Please let me know if the string is limited to show only UTF-8 encoded XML because that works fine.

[RokuJoel]

Hi, I don't know the answer to this offhand, but I would think you should be able to fetch xml in UTF-16 without trouble. (using the XML after you fetch it - that might be a problem)

Can you try to use cURL from the command line to fetch it and see if you are able to get the UTF-16 string that way? If not, there may be something else going on that has nothing to do with the UTF-16 .Also try using the address bar of a web browser.

- Joel

[EnTerr]

First, the following is a guess-work since i have not tried it. But based on circumstantial evidence, i suspect Roku won't be able to handle UTF-16 text string at all. I expect ifUrlTransfer.GetToString() will work only in the sense it will download UTF-16 resource but will return string that is truncated. OTOH, ifUrlTransfer.GetToFile() will likely store the data correct to file but then when you try using ReadAsciiFile(), result will be same junk.

Why? A character is encoded in two bytes in UTF-16, so say a "<" = chr(60) character will be represented as either \x00 \x3c or \x3c \x00, depending on byte order. In C (in which Roku software is written) however, \x00 is used traditionally as end-of-string marker. This explains why you see only one "<" as contents - it must be that transfer was done in UTF-16LE.

What to do? Simple - always use UTF-8 encoding, it is universally used and much more compatible.

[ratish]

@EnTerr I got the same observation that you shared. Roku's String do not support UTF-16. The Data comes in little endian order and hence the String truncates the data after seeing the second byte representing < in UTF-16 encoding.
I got the data into a file and checked the same using roByteArray.