Roku Developer Program

[greubel]

Cleaner ! Signs were wrong on whiles and mantissa didn't compute ?
Knew that if we got you involved (EnTerr), could get some tight code !

   sign = sgn(in)
   x = abs(in)
   r = x
   exp = 0
   while x >= 2: x = x / 2: exp = exp + 1: end while    ' <----
   while x < 1: x = x * 2: exp = exp - 1 : end while    ' <----
'   mant = int((x - 1) * 2^23 + 0.5)                    ' <----
   r = r / ( 2.0 ^ (exp+1) )
   mant = Int( r * (2.0 ^ 24) )

[EnTerr]

"greubel" wrote:
Signs were wrong on whiles and mantissa didn't compute ?

Yes, it was just back-of-the-napkin scribble to show the idea. In addition I missed the case of x=0, when one of the whiles will loop infinitely. In your calculations though, i am a bit concerned i don't see 1 being subtracted? If i remember, there is always implied 1 in front of the 23 fraction bits. Are you testing against real binary floats, besides your own "to" and "from" functions being complementary.

[greubel]

Are you testing against real binary floats ?

Yes, I'm generating a real float, with random values, exponent and sign. Then sending it through the float2byte and then the byte2float and comparing the input value to the result.
One thing, need to start the exp initial value closer to the actual by using the log() function. I'm old school, need efficient code. The way it is now, we could do the whiles a hundred or more times.

[EnTerr]

"greubel" wrote:
Yes, I'm generating a real float, with random values, exponent and sign. Then sending it through the float2byte and then the byte2float and comparing the input value to the result.

No, i meant testing only the two functions against each other won't suffice (consider implementer bias, big vs little endianness). Real, external data like so:

>>> from struct import pack
>>> for f in 0, 1, -1, 123, 0.1, -4.2e9, 3e-42: print pack('f', f).encode('hex'), f
... 
00000000 0
0000803f 1
000080bf -1
0000f642 123
cdcccc3d 0.1
ea567acf -4200000000.0
5d080000 3e-42

One thing, need to start the exp initial value closer to the actual by using the log() function. I'm old school, need efficient code. The way it is now, we could do the whiles a hundred or more times.

Hm, i did not think about that. Yep, could go a good hundred times. You know what, feels trivial - to just take what you were doing as first approximation, exp = int(log(X) / log(2)) : X = X / 2^exp - and then do the whiles. Ha! I bet they will loop only once at most - but even if log/log was off by more, they will get the right result. Brilliant.

[greubel]

OK, back to the original post. WHY DOES THIS FAIL ?

val# = 136103303.90528
? "val = " val# " int = " int(val#) " fix = " fix(val#)
val = 136103303.90528 int = 136103296 fix = 136103296

I can see INT() rounding up by ONE but this answer is off by 7 !
And why doesn't FIX() work ? It always gives the same as int() ????

[EnTerr]

"greubel" wrote:
OK, back to the original post. WHY DOES THIS FAIL ?
I can see INT() rounding up by ONE but this answer is off by 7 !

val# = 136103303.90528
? "val = " val# " int = " int(val#) " fix = " fix(val#)
val = 136103303.90528 int = 136103296 fix = 136103296

Because fix() and int() take Float as argument, not Double. Thus val# gets demoted to a single-precision Float, which has 6+ digits precision. Tough luck.

BrightScript Debugger> ff = function(x as float): return x: end function
BrightScript Debugger> ? type(val#); val#; "becomes "; type(ff(val#)); ff(val#)
Double 136103303.90528 becomes Float 1.36103e+08 

And why doesn't FIX() work ? It always gives the same as int() ????

Fix() and Int() are working correctly, given the maimed float. For positive arguments, fix(x) == int(x).

[greubel]

EnTerr, I PM'ed you some code.