Roku Developer Program

Join our online forum to talk to Roku developers and fellow channel creators. Ask questions, share tips with the community, and find helpful resources.
cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Komag
Roku Guru

Str(value as Float) as String sometimes inaccurate!

Previous, I have code to add up weights of things, and the current total is 8.6, then this:
	? "weight is " weight
A = Str(weight)
? "A is " A

but in the console it prints:
weight is 8.6
A is 8.599999

But when I test just setting beforehand:
weight = 8.6
Then both lines print "8.6" just fine

What the heck is going on??? :?
0 Kudos
11 REPLIES 11
Komag
Roku Guru

Re: Str(value as Float) as String sometimes inaccurate!

A couple extra notes:
Same results on my Roku 2 XS 3100X and Roku HD N1100

The values that were added up to make 8.6 are:
1, 2, 1, 0.1, 0.2, 0.2, 0.1, 1, 2, 1
So there is no fancy division or anything like that, should be exact and straightforward.
0 Kudos
RokuKC
Roku Employee
Roku Employee

Re: Str(value as Float) as String sometimes inaccurate!

"Komag" wrote:
The values that were added up to make 8.6 are:
1, 2, 1, 0.1, 0.2, 0.2, 0.1, 1, 2, 1
So there is no fancy division or anything like that, should be exact and straightforward.


0.1 is not exactly representable in a floating point number, which internally uses a binary representation.

https://docs.python.org/2/tutorial/floatingpoint.html is an article I stumbled across which has some straightforward explanations.

The reason that Str and ? show different results is they happen to use slightly different values for the display precision.
In one case the extra precision shows a more accurate value, whereas the other rounds the small epsilon to show the expected value.
0 Kudos
Komag
Roku Guru

Re: Str(value as Float) as String sometimes inaccurate!

Hmm, okay. I guess I can work around it 8-)
0 Kudos
Rek
Visitor

Re: Str(value as Float) as String sometimes inaccurate!

"RokuKC" wrote:
The reason that Str and ? show different results is they happen to use slightly different values for the display precision.


This obviously leads to confusion... Maybe it's worth defaulting both methods to the same precision, and adding an optional parameter to str() to define precision?
0 Kudos
belltown
Roku Guru

Re: Str(value as Float) as String sometimes inaccurate!

It's not just Str and ? that have issues, but Int() as well:


BrightScript Debugger> a=12345.9996
BrightScript Debugger> ?Int(a)
12346
BrightScript Debugger> a=12345.9995
BrightScript Debugger> ?Int(a)
12345
BrightScript Debugger>

You'd think that Int() would always round down, but I guess that 12345.9996 is stored internally as 123456 as a Float, which is only guaranteed to have 7 digits of precision (10 if it were a Double). Whatever precision you use in this case, there will be some point at which a number will be rounded up when it should be rounded down.
0 Kudos
squirreltown
Roku Guru

Re: Str(value as Float) as String sometimes inaccurate!

I'm guessing fix() will have this issue too, which gives me the opportunity to ask what is the difference between fix() and int() and why isn't there one that rounds up? I can't possibly be the first one who needed that.
Kinetics Screensavers
0 Kudos
Rek
Visitor

Re: Str(value as Float) as String sometimes inaccurate!

"belltown" wrote:
You'd think that Int() would always round down


Apparently so did Roku (http://sdkdocs.roku.com/display/sdkdoc/BrightScript+Language+Reference#BrightScriptLanguageReference...😞

Returns an integer representation of the argument, using the largest whole number that is not greater than the argument


Whatever nonsense caused int(12345.9996) to be rounded up needs to be resolved.
0 Kudos
Rek
Visitor

Re: Str(value as Float) as String sometimes inaccurate!

"squirreltown" wrote:
I'm guessing fix() will have this issue too, which gives me the opportunity to ask what is the difference between fix() and int()


According to the docs on the fix function (which are suspect at this point...):

Returns a truncated representation of the argument. All digits to the right of the decimal point are simply chopped off, so the resultant value is an integer. For non-negative X, FIX(X)=lNT(X). For negative values of X, FIX(X)=INT(X)+1. For example, FIX(2.2) returns 2, and FIX(-2.2) returns -2.


So, the difference should be that int(-2.5) will return -3 since -2 would be greater than -2.5, while fix(-2.5) would return -2 (decimal truncated).

"squirreltown" wrote:
and why isn't there one that rounds up? I can't possibly be the first one who needed that.


The cint() function will round up or down depending on the value ( cint( 2.1 ) -> 2, cint( 2.8 ) -> 3. If you need a function which is guaranteed to always round up (like ceil() in other languages), you'll need to implement it yourself 😞
0 Kudos
RokuMarkn
Visitor

Re: Str(value as Float) as String sometimes inaccurate!

A ceiling function was posted here a while ago: viewtopic.php?f=34&t=51230&p=347768


Whatever nonsense caused int(12345.9996) to be rounded up needs to be resolved.


belltown explained this. The problem isn't in the int function, it's in the value that's being passed to it. 12345.9996 looks like it's different than 12346 but it's not. They both compile to the same float value, since there are only about 7 digits of precision in a float.

--Mark
0 Kudos