Would rounding help?
The problem is pretty well explained in that mailing, but surely,
when we are doing arithmetic to two or three decimal places, there
will be some way to ask that the result be rounded to that many
places. As long as the numbers remain close to 1, it may be easier to
deal with that than forcing the calculations into rational arithmetic.
I have observed rooms full of professors and hardware designers
arguing over these details for months on end, eventually producing
fifty to one hundred page documents to say how it should work. See
IEEE-754 for a serious definition of a good computer arithmetic. It
is indeed very hard to cover all the cases and make it useful as well.
Dick
On 2008, Feb 17, in
[hidden email]:
>>> 1 - 0.2 -0.05 -0.3 -0.1= 0.35 "true"
>>> 1 - 0.2 -0.05 -0.3 -0.1 - 0.10= 0.25 "*****false*****"
>>> 1 - 0.2 -0.05 -0.3 -0.1 - 0.10= 0.24999999999999995 true
...
>>> Can somebody explain that or is it a bug ? I don't know yet if it's
>>> specific to my image, so does other have the same behavior?
>>> I can force the rounding of the result but find it strange.
_______________________________________________
Beginners mailing list
[hidden email]
http://lists.squeakfoundation.org/mailman/listinfo/beginners
Locked
