floating point display errors?

Archive of the old editing forum
Forum rules
Before asking on how to use a ZDoom feature, read the ZDoom wiki first. This forum is archived - please use this set of forums to ask new questions.
Locked
User avatar
Apothem
Posts: 2070
Joined: Sat Nov 29, 2003 7:13 pm
Location: Performing open heart surgery on an ACS compiler.

floating point display errors?

Post by Apothem »

when attempting to display a simple fixed point number, the displayed number via log(); doesnt display correctly.

Code: Select all

int mytestarray[10];
Script 1 enter
{
int myfloat = 1.1;
int myint = 1;
int mycontrol = 1.0;
mytestarray[1] = myfloat;
mytestarray[2] = myint;
mytestarray[3] = mycontrol;
log(s:"myfloat - ", i:myfloat, s:" - ", f:myfloat);
log(s:"myint - ", i:myint, s:" - ", f:myint);
log(s:"mycontrol - ", i:mycontrol, s:" - ", f:mycontrol);
log(s:"testarray ints - ", i:mytestarray[1],s:" - ", i:mytestarray[2],s:" - ", i:mytestarray[3]);
log(s:"testarray floats - ", f:mytestarray[1],s:" - ", f:mytestarray[2],s:" - ", f:mytestarray[3]);
}
myfloat comes out as 1.09999 under the log(...f:myfloat); - Is this just an issue with displaying the value or does this affect a lot more than just displaying the number?

If this error comes up in more places it might explain as to why I've been having issues with capturing mouse coords after they've been converted to floating point numbers.

More than anything else, I've been running this just as a simple test to see if arrays can hold floating point numbers. It seems like they can, but I'm concerned about this display error. Can anyone provide any insight?
disposable_username2
Posts: 168
Joined: Tue Mar 08, 2011 1:25 pm

Re: floating point display errors?

Post by disposable_username2 »

ACS has no floating point number. Only fixed point ones.

I assume you read http://zdoom.org/wiki/Fixed_point . Fixed point numbers are actually integers, so you can put them in an array.
It's also easy to see that 1.1 cannot be exactly represented as fixed point number.
User avatar
ibm5155
Posts: 1268
Joined: Wed Jul 20, 2011 4:24 pm
Contact:

Re: floating point display errors?

Post by ibm5155 »

disposable_username2 wrote:ACS has no floating point number. Only fixed point ones.

I assume you read http://zdoom.org/wiki/Fixed_point . Fixed point numbers are actually integers, so you can put them in an array.
It's also easy to see that 1.1 cannot be exactly represented as fixed point number.
/\

The good part it´s near to the floating point value.
1.1 is simple
1 -> 65536
.1 ->65536/10 -> 6553
add 65536 + 6553 and the result will be : 72089
if you divide 72089 by 65536 you ´ll get : 1,099990844726525
User avatar
Graf Zahl
Lead GZDoom+Raze Developer
Lead GZDoom+Raze Developer
Posts: 49223
Joined: Sat Jul 19, 2003 10:19 am
Location: Germany

Re: floating point display errors?

Post by Graf Zahl »

Similarly, if you ever checked Doom's armor you may have noticed this:

Armor.Savepercent 33.335

So why 33.335 if it's supposed to be 1/3 (= 33.33333333333333333...)?
Simple: 0.33335 is converted to the next representable fixed point number larger than 0.33333333333333333...
Setting Savepercent to 33.333333333333333 will result in a value that's too small and cause rounding errors.

Lesson: If you do floating/fixed point math, always be aware of inaccuracies caused by numbers that aren't precisely representable.
User avatar
ibm5155
Posts: 1268
Joined: Wed Jul 20, 2011 4:24 pm
Contact:

Re: floating point display errors?

Post by ibm5155 »

the 33.3333... value wouldn´t cause a memory overflow? I can´t imagine how to much work is done to avoid that problem
User avatar
GooberMan
Posts: 1336
Joined: Fri Aug 08, 2003 12:57 am
Location: Helsinki, Finland

Re: floating point display errors?

Post by GooberMan »

ibm5155 wrote:memory overflow
I do not think that term means what you think it means.
User avatar
ibm5155
Posts: 1268
Joined: Wed Jul 20, 2011 4:24 pm
Contact:

Re: floating point display errors?

Post by ibm5155 »

It´s like I´m trying to put a int valua on a byte var
User avatar
Graf Zahl
Lead GZDoom+Raze Developer
Lead GZDoom+Raze Developer
Posts: 49223
Joined: Sat Jul 19, 2003 10:19 am
Location: Germany

Re: floating point display errors?

Post by Graf Zahl »

He's correct, you don't understand what the term means! :P
User avatar
BloodyAcid
Posts: 372
Joined: Thu Jul 26, 2012 10:28 pm
Location: Canadia

Re: floating point display errors?

Post by BloodyAcid »

ibm5155 wrote:It´s like I´m trying to put a int valua on a byte var
That's just wrong data type.
Locked

Return to “Editing (Archive)”