# Roku Developer Program

Developers and content creators—a complete solution for growing an audience directly.
cancel
Showing results for
Show  only  | Search instead for
Did you mean:
Highlighted
Level 10

## Issue with math: float calculation, ToStr()

Try this:
? 1.1* 9
console prints 9.9

But try this:
? str(1.1 * 9)
console prints 9.900001

I assume this is some sort of floating point rounding error having to do with binary math and how computers work, etc etc. But it's a problem for me, because in my game when the player is holding 9 spider abdomens in his or her inventory, on the list the weight is showing up at 9.900001, when everything else is just one decimal point, not six!

Any way to force a float to only be one decimal point accurate?

Is my only recourse to always check the final string and count characters after the "." ?
7 Replies
Highlighted
Level 10

## Re: Issue with math: float calculation, ToStr()

I just implemented this to brute force solve it:

`FUNCTION checkStrFloat(Str AS STRING) AS STRING ' trig by getItemDscr(1), c2Inventory(1)	' Search for a . in the string, then check if the string is longer than one character past the . and if so, trim off the rest	StrLen = Str.Len()	FOR i = 1 TO StrLen		chr = Str.Mid(i -1, 1)								' String.Mid() is 0 based, not 1 based, so need to -1		IF chr = "."																				' Ex: "9.900001", i will be 2			IF StrLen > i +1																		' If there is more than just one character after the decimal point				StrNEW = Left(Str, i+1)						' Left() is 1 based		' Ex: i +1 will be 3, thus first three characters of "9.900001" will be "9.9", one character past the "."				? "checkStrFloat() found long float Str, orig " Str ", shortened " StrNew				RETURN StrNEW			END IF		END IF	END FOR	RETURN StrEND FUNCTION`
Highlighted
Level 8

## Re: Issue with math: float calculation, ToStr()

Due to the way floating point numbers are implemented, your number could be "9.899999", in which case your function will return "9.8" when it should return "9.9".

I'd do something like this (assuming your numbers are stored internally as Doubles, but you want to print them to one significant decimal place):

`function convert(f as double) as string  a = CInt(f * 10)  n = a \ 10  m = abs(a) mod 10  return n.toStr() + "." + m.toStr()end function`
https://github.com/belltown/
Highlighted Roku Employee

## Re: Issue with math: float calculation, ToStr()

"Komag" wrote:
Try this:
? 1.1* 9
console prints 9.9

But try this:
? str(1.1 * 9)
console prints 9.900001

A couple of options:
`x=1.1 * 9' 1) use Fix to try to force 1 digit of precision.  may or may not work?? Str(Fix(x * 10)/10)' 2) use printf-style conversion? x.ToStr("%3.1f")' Note that Str prefixes a blank on non-negative numbers, the above ToStr does not.' if you want that, use x.ToStr("% 3.1f")`
Highlighted
Level 19

## Re: Issue with math: float calculation, ToStr()

My advice is to only use integer math unless there's a compelling reason not to (which there never is). If you only want one decimal point of precision then multiply everything by ten (9 spiders at 11 tenths per abdomen is 99 tenths). There's any number of ways to display 99 tenths as 9.9.

-JT
Highlighted
Level 9

## Re: Issue with math: float calculation, ToStr()

"renojim" wrote:
My advice is to only use integer math unless there's a compelling reason not to (which there never is).

Animation might be an exception.
Kinetics Screensavers
Highlighted
Level 10

## Re: Issue with math: float calculation, ToStr()

For some reason I had the impression that x.ToStr() was only for integers, not floats. Because that actually prints correctly:
`? str(1.1*9) 9.900001? (1.1*9).tostr()9.9`

Thanks for the suggestions. I like the idea of always doing all math with integers, but at this point I don't want to go back and rework everything, so I'm going to go with "laundering" all my floats before converting to strings, with:
`FUNCTION washFloat(dirtyFloat) Int10 = Cint(dirtyFloat *10) cleanFloat = Int10 / 10 RETURN cleanFloatEND FUNCTION`

And then I'll use float.ToStr() for double measure (unless there are Roku models or firmware on which that won't work?)
Highlighted Roku Employee

## Re: Issue with math: float calculation, ToStr()

"Komag" wrote:

And then I'll use float.ToStr() for double measure (unless there are Roku models or firmware on which that won't work?)

Default ToStr() on intrinsic types (numeric, Boolean, and string) goes back to firmware 7.0.
ToStr and string Format with printf format specifiers goes back to firmware 7.6.