Roku Developer Program

[greubel]

Unable to unload a received block of data. I can see the connect, send and response with a network trace. But with the roStreamSocket receive function at line 20 of the snippet, I always get a -1. On line 19, I get a good count of characters in the receive buffer.
Also for some reason if I move lines 9 and 10 above the connect at line 6, the connect fails ???

01   ibuf = createobject("roByteArray")
02   tcp = CreateObject( "roStreamSocket" )
03   adrs = createobject( "roSocketAddress" )
04   adrs.setAddress( host )
05   tcp.setSendToAddress( adrs )
06   tcp.connect()
07   if tcp.isConnected()
08      x = tcp.send( obuf, 0, obuf.Count() )
09      port = createobject("roMessagePort")
10      tcp.setMessagePort( port )
11      tcp.notifyReadable(true)
12      continue = tcp.eOK()
13      x = 0
14      while continue
15         ev = wait( 2000, port )
16         if type(ev) = "roSocketEvent"
17            if ev.getSocketID() = tcp.getID()
18               if tcp.isReadable()
19                  x = tcp.getCountRcvBuf()
20                  y = tcp.receive( ibuf, 0, x )
21                  if y > 0
22                     input = ibuf.ToAsciiString()
23	              end if
24               end if
25            end if
26         end if
27      end while
28   end if

Any help would be appreciated !

[kbenson]

Sounds like you ran into the robytearray allocation bug I reported back in June.

"kbenson's email" wrote:

There's a bug in the way that bytearrays are used with sockets, such
that some byte array methods don't show correct information after it's
been assigned from a socket receive.

For example, count() shows the previous count of the byte array, not
the count after data has been written to is. If it was a newly
initialized ByteArray, that's 0. If it had something else inside
before, that count shows that. The data within the ByteArray is set
correctly from the socket receive method though.

This isn't visible in the example code of mine you corrected, because
when you null terminate the data for conversion to an ASCII string,
adding the zero to null terminate at the right index automatically
sets the correct information for the ByteArray.

This isn't a show-stopper, but it did waste a couple hours of my time
as I tried to figure out why I wasn't getting any data, when in fact I
was but it wasn't accessible without some manual tweaking of the
ByteArray.

In addition, I had to pre-allocate the roByteArray that holds the data, as the method wasn't pre-allocating it.

I have no idea if these are still bugs, this is from code and conversations I haven't touched in over 9 months.

[RokuJoel]

can you send me (via email) your bug report, along with some code that demonstrates the problem, I don't see any bugs in our database that reference this issue.

- Joel

[kbenson]

That content is the bug report, copied whole out of the email I sent to RokuKevin (who I was already in correspondence with regarding the pre-allocation of roByteArray problem I was having) on 2011-06-18.

Neither are show stoppers if you know about them, they are just annoying. Looking at that old code, I have this (received being the byte count received):

    ' Manually tweak bytearray so it sees new data
    inbuffer[received] = 0
    inbuffer[received] = invalid

[kbenson]

"kbenson" wrote:
That content is the bug report, copied whole out of the email I sent to RokuKevin (who I was already in correspondence with regarding the pre-allocation of roByteArray problem I was having) on 2011-06-18.

To be clear, I haven't tested that these are still active bugs. I just mentioned them because they looked similar to the described problem. I won't have time until tomorrow night to do any active testing.

[greubel]

Thanks !
That fixed it. So I guess the bug is still there.

[RokuMEmerson]

The way receive and receiveStr() treat their buffers was deliberate, but how it might be tweaked to be more useful could be discussed.

The receiveStr(Integer size) method uses String as a buffer and is intended to produce a nice shrink-wrapped version of the input. It comes with some restrictions on the actual data since it is stored in a String, and it mallocs its own space for every new read. It also requires the client code to create new space to combine results. The C model is similar to fgets() into a string (without newline termination) or even C++ istream shift operator into a string.

The receive(ByteArray buffer, Integer start, Integer size) method uses ByteArray as a buffer and is intended to be a raw interface that overlays existing data as long as space is available while avoiding copies and other overhead. It doesn't change any of the metadata of the ByteArray. Since the transfer can start in the middle of the buffer, it is unclear whether count() would be meaningful if: a) simply set to the number of received bytes (ignoring start), b) updated to point to the end of the new data so that append() and other array interfaces would work as expected from that point, or c) whether count() should include the possibility that the previous data was already past than the newly read data and so count() wouldn't be updated at all. Since receive() returns the number of bytes transferred to the buffer, client code has all the info to manage the storage even without jumping through hoops with the ByteArray metadata. The C model is similar to fread() into a raw buffer malloced by the client code regardless of other metadata also managed by the client code.

Given the fixed restrictions on using String as a buffer, it seemed natural to give all the "rawness" possible to the interface that uses ByteArray.

[kbenson]

I don't think it's appropriate to take a data structure explicitly exported as a BrightScript component and expect developers to treat it like a different construct based on what you pass it to. The interface is well defined. I would expect the ByteArray's count to be the highest addressable byte offset that was written to, as I think anyone else who reads the docs for that component would as well. It's no less raw to make sure BrightScript behaves as expected when accessing the data, when it doesn't alter the data in any way.

Ignoring principle of least surprise, an argument can still be made that it's worth setting the data correctly in the receive() call as it's much faster than if you manually fix in BrightScript (assuming you need it to report the length correctly). BrightScript is quite slow compared to C/C++, so any code required to make the structure correct after a call to a component written in C/C++ should be done within that component.

Now, I don't want to imply I'm unhappy with what we have now. They networking components are functional for what I used them for, so any argument I have is pedantic at best. I was able to basically forget about the problem and move on once I figured out the workaround.

[RokuMEmerson]

The reasons for updating the ByteArray metadata sound reasonable. It's basically option c.

A misstatement I made changes things a bit, however. The socket receive() doesn't use the current underlying size of the ByteArray as a limit; it uses the current count(), because that's what's available from the internal interface to the underlying brightscript structure. Since it's limited by it, receive() can't change count(), so the current behavior is compliant with the ByteArray interface, at least with respect to count().

It would take a bit of re-engineering of the internal interfaces to ByteArray to do this, but to utilize the ByteArray interface fully via receive(), it should respect the auto-resize flag and re-allocate if allowed and necessary. Then the arrays wouldn't have to be artificially touched by scripts to get them sized. As was pointed out, having the internal implementation do this wouldn't be any less efficient than having the script do it.

[RokuMEmerson]

On the connect issue of the original post in this thread:
Setting its message port is what changes a socket from sync to async. Moving that setting before the connect() makes the connect() async, and so it returns immediately. It doesn't fail; it is just pending. Setting notifyWritable() and waiting on the port until the socket isWritable() or isConnected() would indicate when the connect eventually succeeds.