# Scaled decimals comparisons Classic List Threaded 10 messages Open this post in threaded view
|

## Scaled decimals comparisons

 Hi, I was doing some basic calculations based on a formula, and I wanted to convert the result to a scaled decimal in order to avoid having these "loose" decimals in 10th position or something similar. So I did the following: 82 - (2 * 35.9) - (0 / 2) * (113/121) asScaledDecimal: 1 -> 9.5s1 But When I do this in a test: (82 - (2 * 35.9) - (0 / 2) * (113/121) asScaledDecimal: 1) = 9.5s1 It fails because the comparison returns false. I guess this is the proper behavior, but I'd expected that the conversion from a Float to a scaled decimal would have eliminated the extra precision from the formula. I can do a roundTo: 0.1 before converting, but I'd like to know what would be the proper way of dealing with this. Thanks! Esteban A. Maringolo
Open this post in threaded view
|

## Re: Scaled decimals comparisons

 As a follow up to this, adding a roundTo: before converting into a ScaledDecimal does not work. (((91 - (2 * 35.9) - (0 / 2) * (113/121))  roundTo: 0.1) asScaledDecimal: 1) = 17.9s1 So how do you compare two ScaledDecimals that _should_ be the same? Regards! Esteban A. Maringolo On Tue, Sep 1, 2020 at 1:07 AM Esteban Maringolo <[hidden email]> wrote: > > Hi, > > I was doing some basic calculations based on a formula, and I wanted > to convert the result to a scaled decimal in order to avoid having > these "loose" decimals in 10th position or something similar. > > So I did the following: > 82 - (2 * 35.9) - (0 / 2) * (113/121) asScaledDecimal: 1 -> 9.5s1 > > But When I do this in a test: > (82 - (2 * 35.9) - (0 / 2) * (113/121) asScaledDecimal: 1) = 9.5s1 > > It fails because the comparison returns false. > > I guess this is the proper behavior, but I'd expected that the > conversion from a Float to a scaled decimal would have eliminated the > extra precision from the formula. > > I can do a roundTo: 0.1 before converting, but I'd like to know what > would be the proper way of dealing with this. > > Thanks! > > Esteban A. Maringolo
Open this post in threaded view
|

## Re: Scaled decimals comparisons

 Interesting. Your expression evaluates to true in VA Smalltalk. Am 01.09.20 um 06:17 schrieb Esteban Maringolo: > As a follow up to this, adding a roundTo: before converting into a > ScaledDecimal does not work. > > (((91 - (2 * 35.9) - (0 / 2) * (113/121))  roundTo: 0.1) > asScaledDecimal: 1) = 17.9s1 > > So how do you compare two ScaledDecimals that _should_ be the same? > > Regards! > > Esteban A. Maringolo > > On Tue, Sep 1, 2020 at 1:07 AM Esteban Maringolo <[hidden email]> wrote: >> Hi, >> >> I was doing some basic calculations based on a formula, and I wanted >> to convert the result to a scaled decimal in order to avoid having >> these "loose" decimals in 10th position or something similar. >> >> So I did the following: >> 82 - (2 * 35.9) - (0 / 2) * (113/121) asScaledDecimal: 1 -> 9.5s1 >> >> But When I do this in a test: >> (82 - (2 * 35.9) - (0 / 2) * (113/121) asScaledDecimal: 1) = 9.5s1 >> >> It fails because the comparison returns false. >> >> I guess this is the proper behavior, but I'd expected that the >> conversion from a Float to a scaled decimal would have eliminated the >> extra precision from the formula. >> >> I can do a roundTo: 0.1 before converting, but I'd like to know what >> would be the proper way of dealing with this. >> >> Thanks! >> >> Esteban A. Maringolo > -- ----------------------------------------------------------------------- Objektfabrik Joachim Tuchel          mailto:[hidden email] Fliederweg 1                         http://www.objektfabrik.deD-71640 Ludwigsburg                  http://joachimtuchel.wordpress.comTelefon: +49 7141 56 10 86 0         Fax: +49 7141 56 10 86 1
Open this post in threaded view
|

## Re: Scaled decimals comparisons

 In reply to this post by Esteban A. Maringolo Dolphin shows the same behaviour. I added the following method to ScaledDecimal to help: roundedByScale         | scaleFactor |         scaleFactor := 10 ** scale.         ^(ScaledDecimal newFromNumber: (fraction * scaleFactor) rounded scale: scale) / scaleFactor Translating slightly for Pharo: roundedByScale         | scaleFactor |         scaleFactor := 10 ** scale.         ^(ScaledDecimal newFromNumber: (self * scaleFactor) rounded scale: scale) / scaleFactor You can then do: (82 - (2 * 35.9) - (0 / 2) * (113/121) asScaledDecimal: 1) roundedByScale = 9.5s1  "true" HTH. John > On 1 Sep 2020, at 05:17, Esteban Maringolo <[hidden email]> wrote: > > As a follow up to this, adding a roundTo: before converting into a > ScaledDecimal does not work. > > (((91 - (2 * 35.9) - (0 / 2) * (113/121))  roundTo: 0.1) > asScaledDecimal: 1) = 17.9s1 > > So how do you compare two ScaledDecimals that _should_ be the same? > > Regards! > > Esteban A. Maringolo > > On Tue, Sep 1, 2020 at 1:07 AM Esteban Maringolo <[hidden email]> wrote: >> >> Hi, >> >> I was doing some basic calculations based on a formula, and I wanted >> to convert the result to a scaled decimal in order to avoid having >> these "loose" decimals in 10th position or something similar. >> >> So I did the following: >> 82 - (2 * 35.9) - (0 / 2) * (113/121) asScaledDecimal: 1 -> 9.5s1 >> >> But When I do this in a test: >> (82 - (2 * 35.9) - (0 / 2) * (113/121) asScaledDecimal: 1) = 9.5s1 >> >> It fails because the comparison returns false. >> >> I guess this is the proper behavior, but I'd expected that the >> conversion from a Float to a scaled decimal would have eliminated the >> extra precision from the formula. >> >> I can do a roundTo: 0.1 before converting, but I'd like to know what >> would be the proper way of dealing with this. >> >> Thanks! >> >> Esteban A. Maringolo >
Open this post in threaded view
|

## Re: Scaled decimals comparisons

 In reply to this post by jtuchel It evaluates to false in VW too. I guess it is a matter of internal representation, since ScaledDecimals/FixedPoints are a Fraction after all. What is weird, and I'd say wrong (at least from the UX perspective) is that in Pharo (and Dolphin): ((91 - (2 * 35.9) - (0 / 2) * (113/121))  roundTo: 0.1) -> 17.900000000000002 While in VW and VAST they round the float properly: ((91 - (2 * 35.9) - (0 / 2) * (113/121))  roundTo: 0.1) -> 17.9 Regards! Esteban A. Maringolo On Tue, Sep 1, 2020 at 1:24 AM [hidden email] <[hidden email]> wrote: > > Interesting. Your expression evaluates to true in VA Smalltalk. > > > Am 01.09.20 um 06:17 schrieb Esteban Maringolo: > > As a follow up to this, adding a roundTo: before converting into a > > ScaledDecimal does not work. > > > > (((91 - (2 * 35.9) - (0 / 2) * (113/121))  roundTo: 0.1) > > asScaledDecimal: 1) = 17.9s1 > > > > So how do you compare two ScaledDecimals that _should_ be the same? > > > > Regards! > > > > Esteban A. Maringolo > > > > On Tue, Sep 1, 2020 at 1:07 AM Esteban Maringolo <[hidden email]> wrote: > >> Hi, > >> > >> I was doing some basic calculations based on a formula, and I wanted > >> to convert the result to a scaled decimal in order to avoid having > >> these "loose" decimals in 10th position or something similar. > >> > >> So I did the following: > >> 82 - (2 * 35.9) - (0 / 2) * (113/121) asScaledDecimal: 1 -> 9.5s1 > >> > >> But When I do this in a test: > >> (82 - (2 * 35.9) - (0 / 2) * (113/121) asScaledDecimal: 1) = 9.5s1 > >> > >> It fails because the comparison returns false. > >> > >> I guess this is the proper behavior, but I'd expected that the > >> conversion from a Float to a scaled decimal would have eliminated the > >> extra precision from the formula. > >> > >> I can do a roundTo: 0.1 before converting, but I'd like to know what > >> would be the proper way of dealing with this. > >> > >> Thanks! > >> > >> Esteban A. Maringolo > > > > -- > ----------------------------------------------------------------------- > Objektfabrik Joachim Tuchel          mailto:[hidden email] > Fliederweg 1                         http://www.objektfabrik.de> D-71640 Ludwigsburg                  http://joachimtuchel.wordpress.com> Telefon: +49 7141 56 10 86 0         Fax: +49 7141 56 10 86 1 > > >
Open this post in threaded view
|

## Re: Scaled decimals comparisons

 In reply to this post by rko281 Hi John, I was doing a similar rounding in the test assertion, I think your method is better and broadly usable (and maybe convenient for an #equals: implementation). Thanks! Esteban A. Maringolo On Tue, Sep 1, 2020 at 4:33 AM John Aspinall <[hidden email]> wrote: > > Dolphin shows the same behaviour. I added the following method to ScaledDecimal to help: > > roundedByScale > >         | scaleFactor | > >         scaleFactor := 10 ** scale. > >         ^(ScaledDecimal newFromNumber: (fraction * scaleFactor) rounded scale: scale) / scaleFactor > > > Translating slightly for Pharo: > > roundedByScale > >         | scaleFactor | > >         scaleFactor := 10 ** scale. > >         ^(ScaledDecimal newFromNumber: (self * scaleFactor) rounded scale: scale) / scaleFactor > > > You can then do: > > (82 - (2 * 35.9) - (0 / 2) * (113/121) asScaledDecimal: 1) roundedByScale = 9.5s1  "true" > > > HTH. > > John > > > > > > > > > On 1 Sep 2020, at 05:17, Esteban Maringolo <[hidden email]> wrote: > > > > As a follow up to this, adding a roundTo: before converting into a > > ScaledDecimal does not work. > > > > (((91 - (2 * 35.9) - (0 / 2) * (113/121))  roundTo: 0.1) > > asScaledDecimal: 1) = 17.9s1 > > > > So how do you compare two ScaledDecimals that _should_ be the same? > > > > Regards! > > > > Esteban A. Maringolo > > > > On Tue, Sep 1, 2020 at 1:07 AM Esteban Maringolo <[hidden email]> wrote: > >> > >> Hi, > >> > >> I was doing some basic calculations based on a formula, and I wanted > >> to convert the result to a scaled decimal in order to avoid having > >> these "loose" decimals in 10th position or something similar. > >> > >> So I did the following: > >> 82 - (2 * 35.9) - (0 / 2) * (113/121) asScaledDecimal: 1 -> 9.5s1 > >> > >> But When I do this in a test: > >> (82 - (2 * 35.9) - (0 / 2) * (113/121) asScaledDecimal: 1) = 9.5s1 > >> > >> It fails because the comparison returns false. > >> > >> I guess this is the proper behavior, but I'd expected that the > >> conversion from a Float to a scaled decimal would have eliminated the > >> extra precision from the formula. > >> > >> I can do a roundTo: 0.1 before converting, but I'd like to know what > >> would be the proper way of dealing with this. > >> > >> Thanks! > >> > >> Esteban A. Maringolo > > > >
Open this post in threaded view
|

## Re: Scaled decimals comparisons

 In reply to this post by rko281 Hi john you can propose for inclusion in pharo. - Add a nice comment (with executablecomments would be a plus).- Possibly some tests- Send a PR. On 1 Sep 2020, at 09:32, John Aspinall <[hidden email]> wrote:Dolphin shows the same behaviour. I added the following method to ScaledDecimal to help:roundedByScale | scaleFactor | scaleFactor := 10 ** scale. ^(ScaledDecimal newFromNumber: (fraction * scaleFactor) rounded scale: scale) / scaleFactorTranslating slightly for Pharo:roundedByScale | scaleFactor | scaleFactor := 10 ** scale. ^(ScaledDecimal newFromNumber: (self * scaleFactor) rounded scale: scale) / scaleFactorYou can then do:(82 - (2 * 35.9) - (0 / 2) * (113/121) asScaledDecimal: 1) roundedByScale = 9.5s1  "true"HTH.JohnOn 1 Sep 2020, at 05:17, Esteban Maringolo <[hidden email]> wrote:As a follow up to this, adding a roundTo: before converting into aScaledDecimal does not work.(((91 - (2 * 35.9) - (0 / 2) * (113/121))  roundTo: 0.1)asScaledDecimal: 1) = 17.9s1So how do you compare two ScaledDecimals that _should_ be the same?Regards!Esteban A. MaringoloOn Tue, Sep 1, 2020 at 1:07 AM Esteban Maringolo <[hidden email]> wrote:Hi,I was doing some basic calculations based on a formula, and I wantedto convert the result to a scaled decimal in order to avoid havingthese "loose" decimals in 10th position or something similar.So I did the following:82 - (2 * 35.9) - (0 / 2) * (113/121) asScaledDecimal: 1 -> 9.5s1But When I do this in a test:(82 - (2 * 35.9) - (0 / 2) * (113/121) asScaledDecimal: 1) = 9.5s1It fails because the comparison returns false.I guess this is the proper behavior, but I'd expected that theconversion from a Float to a scaled decimal would have eliminated theextra precision from the formula.I can do a roundTo: 0.1 before converting, but I'd like to know whatwould be the proper way of dealing with this.Thanks!Esteban A. Maringolo --------------------------------------------Stéphane Ducasse03 59 35 87 52Assistant: Aurore Dalle FAX 03 59 57 78 50TEL 03 59 35 86 16S. Ducasse - Inria40, avenue Halley, Parc Scientifique de la Haute Borne, Bât.A, Park PlazaVilleneuve d'Ascq 59650France
Open this post in threaded view
|

## Re: Scaled decimals comparisons

 In reply to this post by Esteban A. Maringolo The behaviour of ScaledDecimal in Squeak and Pharo is- different from what the ANSI Smalltalk standard says- different from what other Smalltalks do (not that they agree)- completely different from decimal arithmetic in COBOL, PL/I, or SQL- seriously confusing.What you EXPECT is an exact rational number that is equal to aninteger divided by a power of 10, so that the printedrepresentation faithfully represents the value with no error.What you GET in Pharo is a rational number that PRINTS witha given number of decimal places but that is all.Thus we expect that  x := 10/3 asScaledDecimal: 2.  x asString ==> '3.33s2'  y := x+x+x.  y asString ==> '9.99s2'But no, y asString => '10.00s2'.Here's what the comment in the VW class FixedPoint says:"There are two possible ways to express FixedPoint numbers.  One is as a scaled Integer, but the problem here is that you can lose precision during intermediate calculations.  For example, a property that seems useful is that the calculation (1.000 / 7 * 7) should give you back the number 1.000.For this reason, we have adopted an alternative representation, which is a slight variation on Fraction.  The number is expressed as the ratio of a numerator to a denominator, with the addition of a precision that is used for printing and for rounding.  The number is not usually rounded off to its scale, but if an intermediate result must be rounded to its scale before being used, the messages #roundedToScale and #truncatedToScale may be used."The last sentence in the first paragraph is one I cannot agree with.If I want a calculation where (1/7 * 7) gives me back 1, then I useexact Fractions. If I am using ScaledDecimal, it is because I *want*fixed point decimal numbers, with the properties appropriate to fixedpoint decimal numbers.  Wanting something that I could use to talk todatabases like MariaDB I found that I had to write my own FixedPointclass, only to find that VW called ScaledDecimal FixedPoint.  *Sigh*.Unless and until the current ScaledDecimal is ripped out and buriedin an unmarked grave at a crossroad with a stake through its heart,you may wish to add VW-compatibility methodsroundedToScale   |t|   t := 10 raisedToInteger: scale.   ^(numerator * t / denominator) rounded / t asScaledDecimal: scaletruncatedToScale   |t|   t := 10 raisedToInteger: scale.   ^(numerator * t / denominator) truncated / t asScaledDecimal: scaleand then use  aScaledDecimal roundedToScale = anotherOne roundedToScaleNote that the ANSI standard, which was agreed to by a whole bunch ofSmalltalk experts, says "A converted to a will be a fraction having the same numeric value but having an integernumerator and a denominator which is ten raised tothe power of the ’s scale factor."Try "(10.00s2 / 3) asFraction" in your Smalltalk, and if theresult has a denominator of 3, SOMEONE stuffed up.On Tue, 1 Sep 2020 at 16:08, Esteban Maringolo <[hidden email]> wrote:Hi, I was doing some basic calculations based on a formula, and I wanted to convert the result to a scaled decimal in order to avoid having these "loose" decimals in 10th position or something similar. So I did the following: 82 - (2 * 35.9) - (0 / 2) * (113/121) asScaledDecimal: 1 -> 9.5s1 But When I do this in a test: (82 - (2 * 35.9) - (0 / 2) * (113/121) asScaledDecimal: 1) = 9.5s1 It fails because the comparison returns false. I guess this is the proper behavior, but I'd expected that the conversion from a Float to a scaled decimal would have eliminated the extra precision from the formula. I can do a roundTo: 0.1 before converting, but I'd like to know what would be the proper way of dealing with this. Thanks! Esteban A. Maringolo