Roku Developer Program

[EnTerr]

"Komag" wrote:
Is the reason those last two weren't properly "fixed" have to do with perfectly accurate binary representation being impossible for some decimals?

Mmmm, no.
Close but no cigar (in my best judgement, after carefully re-reading your guess).
Hint: meditate over the int() and fix() signatures (i.e. argument and return types)

[Komag]

Well, don't int and fix take in a float, not a double float, so the extra precision is wasted?

[EnTerr]

"Komag" wrote:
Well, don't int and fix take in a float, not a double float, so the extra precision is wasted?

Yes - the second time it was int() and fix() to be blamed - because of them taking a Float argument, when feeding them a Double it will get down-cast to whatever a Float can hold. And a Float has only 24 bits for the mantissa/significant (to store significant digits). Now 24 bits is exactly how many we needed after the 8-bit shift right - but the conversion Double->Float has also rounded up the number (which generally is the right thing to do), so the precision is 1 bit less (or is it 0.5 bit?), which here was enough to screw up the result

A possible fix for int/fix functions is to make them take Double argument. Or an unspecified type argument and then choose what to do internally.

[Komag]

Maybe you could convert the double to a string, then truncate anything after a ".", then convert back

[EnTerr]

"Komag" wrote:
Maybe you could convert the double to a string, then truncate anything after a ".", then convert back

Convert Double using what, Str(value as Float) as String ? :mrgreen:

I did find solution for the particular case, as seen in the code above. My side-note point was to highlight how delicate it is doing binary shifts with floating points... akin to how hedgehogs copulate*

(*) very, very carefully indeed.