Hello.
I'm having a problem with numbers in squeak, I'm using a 3.10.2 image update #7179 on a vm 3.8.18b1 running on a OSX 10.5 If I open a workspace and evaluate 1203.22 - 1200 I get 3.220000000000027
It's seems that is something working wrong on this. I tried with ScaledDecimal but when I print the numbers it appends a 's2' after it. I'm doing something like ((((1203.22 - 1200)*100) asInteger)/100) asFloat as a workarround, but I think that there may be a better way to work with this problem
Thanks El problema no es mentirse, el problema es creerse _______________________________________________ Beginners mailing list [hidden email] http://lists.squeakfoundation.org/mailman/listinfo/beginners |
On Mon, Jul 13, 2009 at 05:26:58PM -0300, JuanjoE wrote:
> Hello. > I'm having a problem with numbers in squeak, I'm using a 3.10.2 image update > #7179 on a vm 3.8.18b1 running on a OSX 10.5 > > If I open a workspace and evaluate 1203.22 - 1200 I get 3.220000000000027 > > It's seems that is something working wrong on this. > > I tried with ScaledDecimal but when I print the numbers it appends a 's2' > after it. I'm doing something like ((((1203.22 - 1200)*100) asInteger)/100) > asFloat as a workarround, but I think that there may be a better way to work > with this problem Hi, What you are seeing here seems confusing, but it is perfectly normal. The number 1203.22 in your workspace is treated as a floating point number. When you subtract an integer from it, the result is another floating point number. The ...027 is just an artifact of the precision of floats. In Squeak, floating point numbers are represented by class Float, and are internally implemented as double precision float values. If you want unlimited precision, your intuition was right in using a ScaledDecimal instead of a Float. Interestingly, there as been a lot of discussion recently on the squeak-dev and pharo lists about the "right" way to handle this. Treating 1203.22 as a Float by default meets the expectations of most people who do engineering sorts of arithmetic, but is confusing for people who are not familiar with the limits of floating point arithmetic precision. Dave _______________________________________________ Beginners mailing list [hidden email] http://lists.squeakfoundation.org/mailman/listinfo/beginners |
Try searching on "subtractive cancellation" on google for information on
strategies to avoid this kind of thing. One is to treat (a-b) as (a^2 - b^2)/(a+b), which can reduce the relative error in some cases. Unfortunately the exact strategy often depends on the numbers used and the hardware it runs on. > On Mon, Jul 13, 2009 at 05:26:58PM -0300, JuanjoE wrote: >> Hello. >> I'm having a problem with numbers in squeak, I'm using a 3.10.2 image >> update >> #7179 on a vm 3.8.18b1 running on a OSX 10.5 >> >> If I open a workspace and evaluate 1203.22 - 1200 I get >> 3.220000000000027 >> >> It's seems that is something working wrong on this. >> >> I tried with ScaledDecimal but when I print the numbers it appends a >> 's2' >> after it. I'm doing something like ((((1203.22 - 1200)*100) >> asInteger)/100) >> asFloat as a workarround, but I think that there may be a better way to >> work >> with this problem > > Hi, > > What you are seeing here seems confusing, but it is perfectly normal. The > number 1203.22 in your workspace is treated as a floating point number. > When you subtract an integer from it, the result is another floating > point number. The ...027 is just an artifact of the precision of floats. > > In Squeak, floating point numbers are represented by class Float, and > are internally implemented as double precision float values. > > If you want unlimited precision, your intuition was right in using a > ScaledDecimal instead of a Float. Interestingly, there as been a lot > of discussion recently on the squeak-dev and pharo lists about the > "right" way to handle this. Treating 1203.22 as a Float by default > meets the expectations of most people who do engineering sorts of > arithmetic, but is confusing for people who are not familiar with > the limits of floating point arithmetic precision. > > Dave > > _______________________________________________ > Beginners mailing list > [hidden email] > http://lists.squeakfoundation.org/mailman/listinfo/beginners > _______________________________________________ Beginners mailing list [hidden email] http://lists.squeakfoundation.org/mailman/listinfo/beginners |
Free forum by Nabble | Edit this page |