How to translate ID3 PRIV or GEOB to readable strings from HLS live stream?

I made progress, in case anyone is in the same boat.

data = msg.GetInfo()
        if data.PRIV <> invalid
        ? "+++++++++++++++++++++++++++++++"
        ? "+++++    TIMED METADATA   +++++"

            ba = CreateObject("roByteArray")
            ba.FromHExString(data.PRIV)
            strg = ba.ToAsciiString()
            ? strg
         ? "+++++++++++++++++++++++++++++++"
        end if
'will print out something like 

+++++    TIMED METADATA   +++++
www.nielsen.com/X100zdCIGeIlgZnkYj6UvQ==/-NTXplxffzT26A8zx85cow==/DLUCGQsuwoqz1cqXPQA8xhYmQDTfpKUllSnSwH0mLRmQ4FmN5_oyKk1YgRwu_7Go7HwfmZa7ru8LN_PhMUHpHAcY_ztXgCb6rcSpn9JIES4P4COWfiIeY6T3ILuUMUvetbBcX100zdCIGeIlgZnkYj6UvVg_T7JYP0-8OWM=/00000/51385/00
+++++++++++++++++++++++++++++++

GEOB is still giving me headaches. It ends up all our important data is provided as a GEOB frame. However, using the same code, I never get anything more than "?application/json" as the value:

? "+++++    TIMED METADATA   +++++"
        data = msg.GetInfo()
        ?data
       
         if data.GEOB <> invalid

            ba = CreateObject("roByteArray")
            ba.FromHExString(data.GEOB)
            strg = ba.ToAsciiString()
            ? "GEOB"
            ?strg

        end if

prints this in the console:

+++++    TIMED METADATA   +++++
<Component: roAssociativeArray> =
{
    GEOB: "036170706C69636174696F6E2F6A736F6E006B6579732E6A7300416E7661746F7300747970653D63617074696F6E267074733D3532343734343934267061796C6F61643D747325334435323437342E343934253236746578742533444D4144454C45494E452532353230414C42524947485425323532304A4F494E454425323532304D4525323532304F4E253235323026696E73747265616D3D31"
}
GEOB
?application/json
+++++++++++++++++++++++++++++++

I think that's because your GEOB thingy contains null characters. As far as BrightScript is concerned, as soon as it finds a null character in a string, it treats it as the end of the string.

Before you convert to a string you'll have to go through the byte array extracting each null-terminated piece of string data, then convert each of those to a string.

As Belltown says, the hex encoded string represents the binary data of the GEOB frame.
You would have to decode that into components yourself, e.g. using roByteArray.

http://id3.org/id3v2.4.0-frames 4.15. General encapsulated object =>
Text encoding $xx
MIME type <text string> $00
Filename <text string according to encoding> $00 (00)
Content description <text string according to encoding> $00 (00)
Encapsulated object <binary data>

Thanks! You guys rock!
I am now officially out of my depth. Any pointers on how to decompose the byte array into its components?
...off to StackOverflow... 😉

Got it, thanks for the help! I just split the HEX at "00" and run each string through the translation process. Works like a champ.

"RokuKC" wrote:
As Belltown says, the hex encoded string represents the binary data of the GEOB frame.
You would have to decode that into components yourself, e.g. using roByteArray.

Any chance the un-support of \0 in strings can be fixed? As far as i can tell the \0 arcane quirk is not even documented.

I can imagine the (d)effect stemming from ye olde C but string libraries don't suffer from that since... forever. Even easier if rolling-your-own strings - which seems is what the Co. has done, because UTF-8. Shirley, i am serious - i remember the end of last century writing a script engine for a most unusual hardware^ where we had just barebones C to boot - so as part of it i had to write a simple string library supporting \0. It was no BFD.

As a side effect, that will fix the embarrassment of len(str) being O(n)... <wink-wink, nudge-nudge>

(^) Tandem NonStop - a fault-tolerant system used by banks/stock exchanges with no single point of failure (in some configurations having 3 CPUs and 3 banks of RAM working in tandem on the same thing, with the bus hardware arbitrating majority vote every cycle and isolating defective ones) - all components were hot-swappable, no restart; yes - RAM and CPU were replaceable with no down time :twisted:

"EnTerr" wrote:

Any chance the un-support of \0 in strings can be fixed? As far as i can tell the \0 arcane quirk is not even documented.

I think it is unlikely that Roku would add support for embedded chr(0) characters in BrightScript strings.
In my opinion there isn't a valid use case for it. If you need to work with binary data, roByteArray is your best bet.

"RokuKC" wrote:
I think it is unlikely that Roku would add support for embedded chr(0) characters in BrightScript strings.

Hm, let me correct myself - i implied the only way to allow for U+0000 is by re-implementing String type with length_counter field. I was wrong - turns out there is a wickedly clever way to represent \0 in UTF-8 as 0xC0 0x80 as to avoid ever using the dreaded 0x00 octet. See Modified UTF-8.

However, if i try to do that through roByteArray, bizarre things happen:

Brightscript Debugger> ba = createObject("roByteArray"): ba.fromHexString("c080")
Brightscript Debugger> s = ba.toAsciiString(): ? len(s), s, asc(s)
 1              ??               63

What's going on here? Seems like a bug - len() is correct but the rest is whacked?

"EnTerr" wrote:

"RokuKC" wrote:
I think it is unlikely that Roku would add support for embedded chr(0) characters in BrightScript strings.

Hm, let me correct myself - i implied the only way to allow for U+0000 is by re-implementing String type with length_counter field. I was wrong - turns out there is a wickedly clever way to represent \0 in UTF-8 as 0xC0 0x80 as to avoid ever using the dreaded 0x00 octet. See Modified UTF-8.

However, if i try to do that through roByteArray, bizarre things happen:

Brightscript Debugger> ba = createObject("roByteArray"): ba.fromHexString("c080")
Brightscript Debugger> s = ba.toAsciiString(): ? len(s), s, asc(s)
 1              ??               63

What's going on here? Seems like a bug - len() is correct but the rest is whacked?

C080, or any 2-byte sequence starting with c0 or c1, is just not a valid UTF8 sequence. The official standard doesn't allow for "overlong" encodings (representing a character using a 2-byte encoding when that character can be encoded in a single byte). BrightScript is just implementing the official standard.

Regarding your earlier point: "As far as i can tell the \0 arcane quirk is not even documented." -- it's in the roByteArray description in the Component Reference.