Page 1 of 1
floating point display errors?
Posted: Fri Mar 08, 2013 2:04 am
by Apothem
when attempting to display a simple fixed point number, the displayed number via log(); doesnt display correctly.
Code: Select all
int mytestarray[10];
Script 1 enter
{
int myfloat = 1.1;
int myint = 1;
int mycontrol = 1.0;
mytestarray[1] = myfloat;
mytestarray[2] = myint;
mytestarray[3] = mycontrol;
log(s:"myfloat - ", i:myfloat, s:" - ", f:myfloat);
log(s:"myint - ", i:myint, s:" - ", f:myint);
log(s:"mycontrol - ", i:mycontrol, s:" - ", f:mycontrol);
log(s:"testarray ints - ", i:mytestarray[1],s:" - ", i:mytestarray[2],s:" - ", i:mytestarray[3]);
log(s:"testarray floats - ", f:mytestarray[1],s:" - ", f:mytestarray[2],s:" - ", f:mytestarray[3]);
}
myfloat comes out as 1.09999 under the log(...f:myfloat); - Is this just an issue with displaying the value or does this affect a lot more than just displaying the number?
If this error comes up in more places it might explain as to why I've been having issues with capturing mouse coords after they've been converted to floating point numbers.
More than anything else, I've been running this just as a simple test to see if arrays can hold floating point numbers. It seems like they can, but I'm concerned about this display error. Can anyone provide any insight?
Re: floating point display errors?
Posted: Fri Mar 08, 2013 11:54 am
by disposable_username2
ACS has no floating point number. Only fixed point ones.
I assume you read
http://zdoom.org/wiki/Fixed_point . Fixed point numbers are actually integers, so you can put them in an array.
It's also easy to see that 1.1 cannot be exactly represented as fixed point number.
Re: floating point display errors?
Posted: Sat Mar 09, 2013 5:08 am
by ibm5155
disposable_username2 wrote:ACS has no floating point number. Only fixed point ones.
I assume you read
http://zdoom.org/wiki/Fixed_point . Fixed point numbers are actually integers, so you can put them in an array.
It's also easy to see that 1.1 cannot be exactly represented as fixed point number.
/\
The good part it´s near to the floating point value.
1.1 is simple
1 -> 65536
.1 ->65536/10 -> 6553
add 65536 + 6553 and the result will be : 72089
if you divide 72089 by 65536 you ´ll get : 1,099990844726525
Re: floating point display errors?
Posted: Sun Mar 10, 2013 1:59 pm
by Graf Zahl
Similarly, if you ever checked Doom's armor you may have noticed this:
Armor.Savepercent 33.335
So why 33.335 if it's supposed to be 1/3 (= 33.33333333333333333...)?
Simple: 0.33335 is converted to the next representable fixed point number larger than 0.33333333333333333...
Setting Savepercent to 33.333333333333333 will result in a value that's too small and cause rounding errors.
Lesson: If you do floating/fixed point math, always be aware of inaccuracies caused by numbers that aren't precisely representable.
Re: floating point display errors?
Posted: Sun Mar 10, 2013 8:11 pm
by ibm5155
the 33.3333... value wouldn´t cause a memory overflow? I can´t imagine how to much work is done to avoid that problem
Re: floating point display errors?
Posted: Mon Mar 11, 2013 4:33 am
by GooberMan
ibm5155 wrote:memory overflow
I do not think that term means what you think it means.
Re: floating point display errors?
Posted: Tue Mar 12, 2013 5:43 pm
by ibm5155
It´s like I´m trying to put a int valua on a byte var
Re: floating point display errors?
Posted: Wed Mar 13, 2013 1:32 am
by Graf Zahl
He's correct, you don't understand what the term means!
Re: floating point display errors?
Posted: Wed Mar 13, 2013 8:39 pm
by BloodyAcid
ibm5155 wrote:It´s like I´m trying to put a int valua on a byte var
That's just wrong data type.