Issue with math: float calculation, ToStr()
Try this: ? 1.1* 9 console prints 9.9 But try this: ? str(1.1 * 9) console prints 9.900001 I assume this is some sort of floating point rounding error having to do with binary math and how c...
Forum Discussion
Due to the way floating point numbers are implemented, your number could be "9.899999", in which case your function will return "9.8" when it should return "9.9".
I'd do something like this (assuming your numbers are stored internally as Doubles, but you want to print them to one significant decimal place):
I'd do something like this (assuming your numbers are stored internally as Doubles, but you want to print them to one significant decimal place):
function convert(f as double) as string
a = CInt(f * 10)
n = a \ 10
m = abs(a) mod 10
return n.toStr() + "." + m.toStr()
end function
