Hi, I'm preparing a method to write a given Pharo's numerical value into an
external variant of type VT_DECIMAL. The purpose of this type is to retain accuracy, it uses 12 bytes for "mantissa" and two bytes for a sign and a scale (i. e. the position of decimal point). What would be the best approach for the conversion, taken all the possible subclasses of class Number, that is a Float, a Fraction and the Integers, and to keep the accuracy at the same level? For instance, Fraction's numerator and denominator could be directly imputed into VT_DECIMAL's value and scale if denominator is a power of 10. What to do in other cases? A nice description of VT_DECIMAL is here <https://bytecomb.com/vba-internals-decimal-variables-and-pointers-in-depth/> . Best wishes, Tomaz -- Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html |
VT_DECIMAL sounds like a very close match to ScaledDecimal.
Or it would if ScaledDecimal were consistently implemented between Smalltalks. The intent behind ScaledDecimal in the ANSI Smalltalk standard appears to have been an (m,p) representation where m and p are Integers standing for m * 10^p, for interoperation with DECIMAL fields in SQL databases (and of course with legacy code in COBOL and PL/I). This seems clear enough: 'Scaled decimal objects provide a precise representation of decimal fractions with an explicitly specified number of fractional digits.' That's not what VisualWorks, Squeak, and Pharo actually do. Instead they use a (q,p) representation where q is an *arbitrary* Fraction and p just says how many decimal places to *print*. This leads to some very confusing results, and means that the class *invented* to handle decimal fixed point is no more helpful to you in Pharo than Fraction is. The reason VW gives is that it "seems useful" that 1000s0/7 * 7 should equal 1000s0, but that's actually *not* a useful property for an interoperability class. If I want a Fraction, I know where to find it, and if I want it printed to a certain number of decimal places, I know how to do *that* without needing a complete number class for the purpose. VisualAge Smalltalk represents a decimal number as an array of 17 bits, which is a bit more adequate than VT_DECIMAL, and is precisely the IBM mainframe "packed decimal" format with an extra byte for a scale. So what should you do? Basically, a VT_DECIMAL is a pair (m,p,s) where m between: 0 and: (2 raisedTo: 96) - 1, p between: 0 and: 28, and s between: 0 and 1. So Number methods for: 'converting' asVtDecimalParts: p "p is the desired scale" |m s| s := 0. m := (self * (10 raisedToInteger: p)) rounded. m < 0 ifTrue: [s := 1. m := m negated]. ^Array with: m with: p with: s Note that with a Fraction, saying "is the denominator a power of 10, and if so which?" won't work, because 7/5 has a denominator that is not a power of 10 but it is exactly representable as 1.4s1. And in both ANSI Smalltalk and VT_DECIMAL, the numbers 1.4s1, 1.40s2, 1.400s3, and so on are distinguishable. So the scale *has* to be something over and above the numeric value. To put it another way, the scale is determined by what the external application WANTS, not what the Smalltalk code HAS. On Wed, 9 Oct 2019 at 09:24, eftomi <[hidden email]> wrote: > > Hi, I'm preparing a method to write a given Pharo's numerical value into an > external variant of type VT_DECIMAL. The purpose of this type is to retain > accuracy, it uses 12 bytes for "mantissa" and two bytes for a sign and a > scale (i. e. the position of decimal point). What would be the best approach > for the conversion, taken all the possible subclasses of class Number, that > is a Float, a Fraction and the Integers, and to keep the accuracy at the > same level? > > For instance, Fraction's numerator and denominator could be directly imputed > into VT_DECIMAL's value and scale if denominator is a power of 10. What to > do in other cases? > > A nice description of VT_DECIMAL is here > <https://bytecomb.com/vba-internals-decimal-variables-and-pointers-in-depth/> > . > > Best wishes, > Tomaz > > > > -- > Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html > |
Whoops, Visual Age uses 17 BYTES, not 17 BITS. Slaps own wrist.
On Wed, 9 Oct 2019 at 22:03, Richard O'Keefe <[hidden email]> wrote: > > VT_DECIMAL sounds like a very close match to ScaledDecimal. > Or it would if ScaledDecimal were consistently implemented between Smalltalks. > The intent behind ScaledDecimal in the ANSI Smalltalk standard appears to > have been an (m,p) representation where m and p are Integers standing for > m * 10^p, for interoperation with DECIMAL fields in SQL databases (and of > course with legacy code in COBOL and PL/I). This > seems clear enough: 'Scaled decimal objects provide a precise representation of > decimal fractions with an explicitly specified number of fractional digits.' > > That's not what VisualWorks, Squeak, and Pharo actually do. Instead > they use a (q,p) > representation where q is an *arbitrary* Fraction and p just says how many > decimal places to *print*. This leads to some very confusing results, and means > that the class *invented* to handle decimal fixed point is no more > helpful to you in > Pharo than Fraction is. The reason VW gives is that it "seems useful" that > 1000s0/7 * 7 should equal 1000s0, but that's actually *not* a useful > property for an > interoperability class. If I want a Fraction, I know where to find > it, and if I want it > printed to a certain number of decimal places, I know how to do *that* without > needing a complete number class for the purpose. > > VisualAge Smalltalk represents a decimal number as an array of 17 bits, which is > a bit more adequate than VT_DECIMAL, and is precisely the IBM mainframe > "packed decimal" format with an extra byte for a scale. > > So what should you do? Basically, a VT_DECIMAL is a pair (m,p,s) where > m between: 0 and: (2 raisedTo: 96) - 1, p between: 0 and: 28, and > s between: 0 and 1. So > > Number > methods for: 'converting' > asVtDecimalParts: p "p is the desired scale" > |m s| > s := 0. > m := (self * (10 raisedToInteger: p)) rounded. > m < 0 ifTrue: [s := 1. m := m negated]. > ^Array with: m with: p with: s > > Note that with a Fraction, saying "is the denominator a power of 10, > and if so which?" > won't work, because 7/5 has a denominator that is not a power of 10 but it is > exactly representable as 1.4s1. And in both ANSI Smalltalk and VT_DECIMAL, > the numbers 1.4s1, 1.40s2, 1.400s3, and so on are distinguishable. So the scale > *has* to be something over and above the numeric value. To put it another way, > the scale is determined by what the external application WANTS, not what the > Smalltalk code HAS. > > On Wed, 9 Oct 2019 at 09:24, eftomi <[hidden email]> wrote: > > > > Hi, I'm preparing a method to write a given Pharo's numerical value into an > > external variant of type VT_DECIMAL. The purpose of this type is to retain > > accuracy, it uses 12 bytes for "mantissa" and two bytes for a sign and a > > scale (i. e. the position of decimal point). What would be the best approach > > for the conversion, taken all the possible subclasses of class Number, that > > is a Float, a Fraction and the Integers, and to keep the accuracy at the > > same level? > > > > For instance, Fraction's numerator and denominator could be directly imputed > > into VT_DECIMAL's value and scale if denominator is a power of 10. What to > > do in other cases? > > > > A nice description of VT_DECIMAL is here > > <https://bytecomb.com/vba-internals-decimal-variables-and-pointers-in-depth/> > > . > > > > Best wishes, > > Tomaz > > > > > > > > -- > > Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html > > |
Thanks for the explanation and solution - it works flawlessly and efficient
:-) -- Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html |
Free forum by Nabble | Edit this page |