Hi,
I was doing some basic calculations based on a formula, and I wanted to convert the result to a scaled decimal in order to avoid having these "loose" decimals in 10th position or something similar. So I did the following: 82 - (2 * 35.9) - (0 / 2) * (113/121) asScaledDecimal: 1 -> 9.5s1 But When I do this in a test: (82 - (2 * 35.9) - (0 / 2) * (113/121) asScaledDecimal: 1) = 9.5s1 It fails because the comparison returns false. I guess this is the proper behavior, but I'd expected that the conversion from a Float to a scaled decimal would have eliminated the extra precision from the formula. I can do a roundTo: 0.1 before converting, but I'd like to know what would be the proper way of dealing with this. Thanks! Esteban A. Maringolo |
As a follow up to this, adding a roundTo: before converting into a
ScaledDecimal does not work. (((91 - (2 * 35.9) - (0 / 2) * (113/121)) roundTo: 0.1) asScaledDecimal: 1) = 17.9s1 So how do you compare two ScaledDecimals that _should_ be the same? Regards! Esteban A. Maringolo On Tue, Sep 1, 2020 at 1:07 AM Esteban Maringolo <[hidden email]> wrote: > > Hi, > > I was doing some basic calculations based on a formula, and I wanted > to convert the result to a scaled decimal in order to avoid having > these "loose" decimals in 10th position or something similar. > > So I did the following: > 82 - (2 * 35.9) - (0 / 2) * (113/121) asScaledDecimal: 1 -> 9.5s1 > > But When I do this in a test: > (82 - (2 * 35.9) - (0 / 2) * (113/121) asScaledDecimal: 1) = 9.5s1 > > It fails because the comparison returns false. > > I guess this is the proper behavior, but I'd expected that the > conversion from a Float to a scaled decimal would have eliminated the > extra precision from the formula. > > I can do a roundTo: 0.1 before converting, but I'd like to know what > would be the proper way of dealing with this. > > Thanks! > > Esteban A. Maringolo |
Interesting. Your expression evaluates to true in VA Smalltalk.
Am 01.09.20 um 06:17 schrieb Esteban Maringolo: > As a follow up to this, adding a roundTo: before converting into a > ScaledDecimal does not work. > > (((91 - (2 * 35.9) - (0 / 2) * (113/121)) roundTo: 0.1) > asScaledDecimal: 1) = 17.9s1 > > So how do you compare two ScaledDecimals that _should_ be the same? > > Regards! > > Esteban A. Maringolo > > On Tue, Sep 1, 2020 at 1:07 AM Esteban Maringolo <[hidden email]> wrote: >> Hi, >> >> I was doing some basic calculations based on a formula, and I wanted >> to convert the result to a scaled decimal in order to avoid having >> these "loose" decimals in 10th position or something similar. >> >> So I did the following: >> 82 - (2 * 35.9) - (0 / 2) * (113/121) asScaledDecimal: 1 -> 9.5s1 >> >> But When I do this in a test: >> (82 - (2 * 35.9) - (0 / 2) * (113/121) asScaledDecimal: 1) = 9.5s1 >> >> It fails because the comparison returns false. >> >> I guess this is the proper behavior, but I'd expected that the >> conversion from a Float to a scaled decimal would have eliminated the >> extra precision from the formula. >> >> I can do a roundTo: 0.1 before converting, but I'd like to know what >> would be the proper way of dealing with this. >> >> Thanks! >> >> Esteban A. Maringolo > -- ----------------------------------------------------------------------- Objektfabrik Joachim Tuchel mailto:[hidden email] Fliederweg 1 http://www.objektfabrik.de D-71640 Ludwigsburg http://joachimtuchel.wordpress.com Telefon: +49 7141 56 10 86 0 Fax: +49 7141 56 10 86 1 |
In reply to this post by Esteban A. Maringolo
Dolphin shows the same behaviour. I added the following method to ScaledDecimal to help:
roundedByScale | scaleFactor | scaleFactor := 10 ** scale. ^(ScaledDecimal newFromNumber: (fraction * scaleFactor) rounded scale: scale) / scaleFactor Translating slightly for Pharo: roundedByScale | scaleFactor | scaleFactor := 10 ** scale. ^(ScaledDecimal newFromNumber: (self * scaleFactor) rounded scale: scale) / scaleFactor You can then do: (82 - (2 * 35.9) - (0 / 2) * (113/121) asScaledDecimal: 1) roundedByScale = 9.5s1 "true" HTH. John > On 1 Sep 2020, at 05:17, Esteban Maringolo <[hidden email]> wrote: > > As a follow up to this, adding a roundTo: before converting into a > ScaledDecimal does not work. > > (((91 - (2 * 35.9) - (0 / 2) * (113/121)) roundTo: 0.1) > asScaledDecimal: 1) = 17.9s1 > > So how do you compare two ScaledDecimals that _should_ be the same? > > Regards! > > Esteban A. Maringolo > > On Tue, Sep 1, 2020 at 1:07 AM Esteban Maringolo <[hidden email]> wrote: >> >> Hi, >> >> I was doing some basic calculations based on a formula, and I wanted >> to convert the result to a scaled decimal in order to avoid having >> these "loose" decimals in 10th position or something similar. >> >> So I did the following: >> 82 - (2 * 35.9) - (0 / 2) * (113/121) asScaledDecimal: 1 -> 9.5s1 >> >> But When I do this in a test: >> (82 - (2 * 35.9) - (0 / 2) * (113/121) asScaledDecimal: 1) = 9.5s1 >> >> It fails because the comparison returns false. >> >> I guess this is the proper behavior, but I'd expected that the >> conversion from a Float to a scaled decimal would have eliminated the >> extra precision from the formula. >> >> I can do a roundTo: 0.1 before converting, but I'd like to know what >> would be the proper way of dealing with this. >> >> Thanks! >> >> Esteban A. Maringolo > |
In reply to this post by jtuchel
It evaluates to false in VW too.
I guess it is a matter of internal representation, since ScaledDecimals/FixedPoints are a Fraction after all. What is weird, and I'd say wrong (at least from the UX perspective) is that in Pharo (and Dolphin): ((91 - (2 * 35.9) - (0 / 2) * (113/121)) roundTo: 0.1) -> 17.900000000000002 While in VW and VAST they round the float properly: ((91 - (2 * 35.9) - (0 / 2) * (113/121)) roundTo: 0.1) -> 17.9 Regards! Esteban A. Maringolo On Tue, Sep 1, 2020 at 1:24 AM [hidden email] <[hidden email]> wrote: > > Interesting. Your expression evaluates to true in VA Smalltalk. > > > Am 01.09.20 um 06:17 schrieb Esteban Maringolo: > > As a follow up to this, adding a roundTo: before converting into a > > ScaledDecimal does not work. > > > > (((91 - (2 * 35.9) - (0 / 2) * (113/121)) roundTo: 0.1) > > asScaledDecimal: 1) = 17.9s1 > > > > So how do you compare two ScaledDecimals that _should_ be the same? > > > > Regards! > > > > Esteban A. Maringolo > > > > On Tue, Sep 1, 2020 at 1:07 AM Esteban Maringolo <[hidden email]> wrote: > >> Hi, > >> > >> I was doing some basic calculations based on a formula, and I wanted > >> to convert the result to a scaled decimal in order to avoid having > >> these "loose" decimals in 10th position or something similar. > >> > >> So I did the following: > >> 82 - (2 * 35.9) - (0 / 2) * (113/121) asScaledDecimal: 1 -> 9.5s1 > >> > >> But When I do this in a test: > >> (82 - (2 * 35.9) - (0 / 2) * (113/121) asScaledDecimal: 1) = 9.5s1 > >> > >> It fails because the comparison returns false. > >> > >> I guess this is the proper behavior, but I'd expected that the > >> conversion from a Float to a scaled decimal would have eliminated the > >> extra precision from the formula. > >> > >> I can do a roundTo: 0.1 before converting, but I'd like to know what > >> would be the proper way of dealing with this. > >> > >> Thanks! > >> > >> Esteban A. Maringolo > > > > -- > ----------------------------------------------------------------------- > Objektfabrik Joachim Tuchel mailto:[hidden email] > Fliederweg 1 http://www.objektfabrik.de > D-71640 Ludwigsburg http://joachimtuchel.wordpress.com > Telefon: +49 7141 56 10 86 0 Fax: +49 7141 56 10 86 1 > > > |
In reply to this post by rko281
Hi John,
I was doing a similar rounding in the test assertion, I think your method is better and broadly usable (and maybe convenient for an #equals: implementation). Thanks! Esteban A. Maringolo On Tue, Sep 1, 2020 at 4:33 AM John Aspinall <[hidden email]> wrote: > > Dolphin shows the same behaviour. I added the following method to ScaledDecimal to help: > > roundedByScale > > | scaleFactor | > > scaleFactor := 10 ** scale. > > ^(ScaledDecimal newFromNumber: (fraction * scaleFactor) rounded scale: scale) / scaleFactor > > > Translating slightly for Pharo: > > roundedByScale > > | scaleFactor | > > scaleFactor := 10 ** scale. > > ^(ScaledDecimal newFromNumber: (self * scaleFactor) rounded scale: scale) / scaleFactor > > > You can then do: > > (82 - (2 * 35.9) - (0 / 2) * (113/121) asScaledDecimal: 1) roundedByScale = 9.5s1 "true" > > > HTH. > > John > > > > > > > > > On 1 Sep 2020, at 05:17, Esteban Maringolo <[hidden email]> wrote: > > > > As a follow up to this, adding a roundTo: before converting into a > > ScaledDecimal does not work. > > > > (((91 - (2 * 35.9) - (0 / 2) * (113/121)) roundTo: 0.1) > > asScaledDecimal: 1) = 17.9s1 > > > > So how do you compare two ScaledDecimals that _should_ be the same? > > > > Regards! > > > > Esteban A. Maringolo > > > > On Tue, Sep 1, 2020 at 1:07 AM Esteban Maringolo <[hidden email]> wrote: > >> > >> Hi, > >> > >> I was doing some basic calculations based on a formula, and I wanted > >> to convert the result to a scaled decimal in order to avoid having > >> these "loose" decimals in 10th position or something similar. > >> > >> So I did the following: > >> 82 - (2 * 35.9) - (0 / 2) * (113/121) asScaledDecimal: 1 -> 9.5s1 > >> > >> But When I do this in a test: > >> (82 - (2 * 35.9) - (0 / 2) * (113/121) asScaledDecimal: 1) = 9.5s1 > >> > >> It fails because the comparison returns false. > >> > >> I guess this is the proper behavior, but I'd expected that the > >> conversion from a Float to a scaled decimal would have eliminated the > >> extra precision from the formula. > >> > >> I can do a roundTo: 0.1 before converting, but I'd like to know what > >> would be the proper way of dealing with this. > >> > >> Thanks! > >> > >> Esteban A. Maringolo > > > > |
In reply to this post by rko281
Hi john you can propose for inclusion in pharo.
- Add a nice comment (with executablecomments would be a plus). - Possibly some tests - Send a PR.
-------------------------------------------- Stéphane Ducasse 03 59 35 87 52 Assistant: Aurore Dalle FAX 03 59 57 78 50 TEL 03 59 35 86 16 S. Ducasse - Inria 40, avenue Halley, Parc Scientifique de la Haute Borne, Bât.A, Park Plaza Villeneuve d'Ascq 59650 France |
In reply to this post by Esteban A. Maringolo
The behaviour of ScaledDecimal in Squeak and Pharo is - different from what the ANSI Smalltalk standard says - different from what other Smalltalks do (not that they agree) - completely different from decimal arithmetic in COBOL, PL/I, or SQL - seriously confusing. What you EXPECT is an exact rational number that is equal to an integer divided by a power of 10, so that the printed representation faithfully represents the value with no error. What you GET in Pharo is a rational number that PRINTS with a given number of decimal places but that is all. Thus we expect that x := 10/3 asScaledDecimal: 2. x asString ==> '3.33s2' y := x+x+x. y asString ==> '9.99s2' But no, y asString => '10.00s2'. Here's what the comment in the VW class FixedPoint says: "There are two possible ways to express FixedPoint numbers. One is as a scaled Integer, but the problem here is that you can lose precision during intermediate calculations. For example, a property that seems useful is that the calculation (1.000 / 7 * 7) should give you back the number 1.000. For this reason, we have adopted an alternative representation, which is a slight variation on Fraction. The number is expressed as the ratio of a numerator to a denominator, with the addition of a precision that is used for printing and for rounding. The number is not usually rounded off to its scale, but if an intermediate result must be rounded to its scale before being used, the messages #roundedToScale and #truncatedToScale may be used." The last sentence in the first paragraph is one I cannot agree with. If I want a calculation where (1/7 * 7) gives me back 1, then I use exact Fractions. If I am using ScaledDecimal, it is because I *want* fixed point decimal numbers, with the properties appropriate to fixed point decimal numbers. Wanting something that I could use to talk to databases like MariaDB I found that I had to write my own FixedPoint class, only to find that VW called ScaledDecimal FixedPoint. *Sigh*. Unless and until the current ScaledDecimal is ripped out and buried in an unmarked grave at a crossroad with a stake through its heart, you may wish to add VW-compatibility methods roundedToScale |t| t := 10 raisedToInteger: scale. ^(numerator * t / denominator) rounded / t asScaledDecimal: scale truncatedToScale |t| t := 10 raisedToInteger: scale. ^(numerator * t / denominator) truncated / t asScaledDecimal: scale and then use aScaledDecimal roundedToScale = anotherOne roundedToScale Note that the ANSI standard, which was agreed to by a whole bunch of Smalltalk experts, says "A <scaledDecimal> converted to a <Fraction> will be a fraction having the same numeric value but having an integer numerator and a denominator which is ten raised to the power of the <scaledDecimal>’s scale factor." Try "(10.00s2 / 3) asFraction" in your Smalltalk, and if the result has a denominator of 3, SOMEONE stuffed up. On Tue, 1 Sep 2020 at 16:08, Esteban Maringolo <[hidden email]> wrote: Hi, |
Richard
we are really interested in improving Pharo on that aspect. Now I’m really not good with such domain. Can you send us some code and tests? S.
-------------------------------------------- Stéphane Ducasse 03 59 35 87 52 Assistant: Aurore Dalle FAX 03 59 57 78 50 TEL 03 59 35 86 16 S. Ducasse - Inria 40, avenue Halley, Parc Scientifique de la Haute Borne, Bât.A, Park Plaza Villeneuve d'Ascq 59650 France |
Hi Richard,
+1 to what was asked by Stef. How hard is it to bring your knowledge, and existing implementation, into a "proper" implementation of ScaledDecimal (or FixedPoint) in Pharo? Regards! Esteban A. Maringolo On Wed, Sep 2, 2020 at 12:12 PM Stéphane Ducasse <[hidden email]> wrote: > > Richard > > we are really interested in improving Pharo on that aspect. > Now I’m really not good with such domain. > Can you send us some code and tests? > > S. > > On 2 Sep 2020, at 12:31, Richard O'Keefe <[hidden email]> wrote: > > The behaviour of ScaledDecimal in Squeak and Pharo is > - different from what the ANSI Smalltalk standard says > - different from what other Smalltalks do (not that they agree) > - completely different from decimal arithmetic in COBOL, PL/I, or SQL > - seriously confusing. > > What you EXPECT is an exact rational number that is equal to an > integer divided by a power of 10, so that the printed > representation faithfully represents the value with no error. > What you GET in Pharo is a rational number that PRINTS with > a given number of decimal places but that is all. > > Thus we expect that > x := 10/3 asScaledDecimal: 2. > x asString ==> '3.33s2' > y := x+x+x. > y asString ==> '9.99s2' > But no, y asString => '10.00s2'. > > Here's what the comment in the VW class FixedPoint says: > "There are two possible ways to express FixedPoint numbers. One is as a scaled Integer, but the problem here is that you can lose precision during intermediate calculations. For example, a property that seems useful is that the calculation (1.000 / 7 * 7) should give you back the number 1.000. > > For this reason, we have adopted an alternative representation, which is a slight variation on Fraction. The number is expressed as the ratio of a numerator to a denominator, with the addition of a precision that is used for printing and for rounding. The number is not usually rounded off to its scale, but if an intermediate result must be rounded to its scale before being used, the messages #roundedToScale and #truncatedToScale may be used." > > The last sentence in the first paragraph is one I cannot agree with. > If I want a calculation where (1/7 * 7) gives me back 1, then I use > exact Fractions. If I am using ScaledDecimal, it is because I *want* > fixed point decimal numbers, with the properties appropriate to fixed > point decimal numbers. Wanting something that I could use to talk to > databases like MariaDB I found that I had to write my own FixedPoint > class, only to find that VW called ScaledDecimal FixedPoint. *Sigh*. > > Unless and until the current ScaledDecimal is ripped out and buried > in an unmarked grave at a crossroad with a stake through its heart, > you may wish to add VW-compatibility methods > > roundedToScale > |t| > t := 10 raisedToInteger: scale. > ^(numerator * t / denominator) rounded / t asScaledDecimal: scale > > truncatedToScale > |t| > t := 10 raisedToInteger: scale. > ^(numerator * t / denominator) truncated / t asScaledDecimal: scale > > and then use > > aScaledDecimal roundedToScale = anotherOne roundedToScale > > Note that the ANSI standard, which was agreed to by a whole bunch of > Smalltalk experts, says "A <scaledDecimal> converted to a <Fraction> > will be a fraction having the same numeric value but having an integer > numerator and a denominator which is ten raised to > the power of the <scaledDecimal>’s scale factor." > Try "(10.00s2 / 3) asFraction" in your Smalltalk, and if the > result has a denominator of 3, SOMEONE stuffed up. > > > > > On Tue, 1 Sep 2020 at 16:08, Esteban Maringolo <[hidden email]> wrote: >> >> Hi, >> >> I was doing some basic calculations based on a formula, and I wanted >> to convert the result to a scaled decimal in order to avoid having >> these "loose" decimals in 10th position or something similar. >> >> So I did the following: >> 82 - (2 * 35.9) - (0 / 2) * (113/121) asScaledDecimal: 1 -> 9.5s1 >> >> But When I do this in a test: >> (82 - (2 * 35.9) - (0 / 2) * (113/121) asScaledDecimal: 1) = 9.5s1 >> >> It fails because the comparison returns false. >> >> I guess this is the proper behavior, but I'd expected that the >> conversion from a Float to a scaled decimal would have eliminated the >> extra precision from the formula. >> >> I can do a roundTo: 0.1 before converting, but I'd like to know what >> would be the proper way of dealing with this. >> >> Thanks! >> >> Esteban A. Maringolo >> > > -------------------------------------------- > Stéphane Ducasse > http://stephane.ducasse.free.fr / http://www.pharo.org > 03 59 35 87 52 > Assistant: Aurore Dalle > FAX 03 59 57 78 50 > TEL 03 59 35 86 16 > S. Ducasse - Inria > 40, avenue Halley, > Parc Scientifique de la Haute Borne, Bât.A, Park Plaza > Villeneuve d'Ascq 59650 > France > |
Free forum by Nabble | Edit this page |