Komag
7 years agoRoku Guru
Issue with math: float calculation, ToStr()
Try this:
? 1.1* 9
console prints 9.9
But try this:
? str(1.1 * 9)
console prints 9.900001
I assume this is some sort of floating point rounding error having to do with binary math and how computers work, etc etc. But it's a problem for me, because in my game when the player is holding 9 spider abdomens in his or her inventory, on the list the weight is showing up at 9.900001, when everything else is just one decimal point, not six!
Any way to force a float to only be one decimal point accurate?
Is my only recourse to always check the final string and count characters after the "." ?
? 1.1* 9
console prints 9.9
But try this:
? str(1.1 * 9)
console prints 9.900001
I assume this is some sort of floating point rounding error having to do with binary math and how computers work, etc etc. But it's a problem for me, because in my game when the player is holding 9 spider abdomens in his or her inventory, on the list the weight is showing up at 9.900001, when everything else is just one decimal point, not six!
Any way to force a float to only be one decimal point accurate?
Is my only recourse to always check the final string and count characters after the "." ?