# Number to VT_DECIMAL

4 messages
Open this post in threaded view
|

## Number to VT_DECIMAL

 Hi, I'm preparing a method to write a given Pharo's numerical value into an external variant of type VT_DECIMAL. The purpose of this type is to retain accuracy, it uses 12 bytes for "mantissa" and two bytes for a sign and a scale (i. e. the position of decimal point). What would be the best approach for the conversion, taken all the possible subclasses of class Number, that is a Float, a Fraction and the Integers, and to keep the accuracy at the same level? For instance, Fraction's numerator and denominator could be directly imputed into VT_DECIMAL's value and scale if denominator is a power of 10. What to do in other cases? A nice description of VT_DECIMAL is  here . Best wishes, Tomaz -- Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
Open this post in threaded view
|

## Re: Number to VT_DECIMAL

 VT_DECIMAL sounds like a very close match to ScaledDecimal. Or it would if ScaledDecimal were consistently implemented between Smalltalks. The intent behind ScaledDecimal in the ANSI Smalltalk standard appears to have been an (m,p) representation where m and p are Integers standing for m * 10^p, for interoperation with DECIMAL fields in SQL databases (and of course with legacy code in COBOL and PL/I). This seems clear enough: 'Scaled decimal objects provide a precise representation of decimal fractions with an explicitly specified number of fractional digits.' That's not what VisualWorks, Squeak, and Pharo actually do.  Instead they use a (q,p) representation where q is an *arbitrary* Fraction and p just says how many decimal places to *print*.  This leads to some very confusing results, and means that the class *invented* to handle decimal fixed point is no more helpful to you in Pharo than Fraction is.  The reason VW gives is that it "seems useful" that 1000s0/7 * 7 should equal 1000s0, but that's actually *not* a useful property for an interoperability class.  If I want a Fraction, I know where to find it, and if I want it printed to a certain number of decimal places, I know how to do *that* without needing a complete number class for the purpose. VisualAge Smalltalk represents a decimal number as an array of 17 bits, which is a bit more adequate than VT_DECIMAL, and is precisely the IBM mainframe "packed decimal" format with an extra byte for a scale. So what should you do?  Basically, a VT_DECIMAL is a pair (m,p,s) where m between: 0 and: (2 raisedTo: 96) - 1, p between: 0 and: 28, and s between: 0 and 1.  So Number   methods for: 'converting'     asVtDecimalParts: p "p is the desired scale"       |m s|       s := 0.       m := (self * (10 raisedToInteger: p)) rounded.       m < 0 ifTrue: [s := 1. m := m negated].       ^Array with: m with: p with: s Note that with a Fraction, saying "is the denominator a power of 10, and if so which?" won't work, because 7/5 has a denominator that is not a power of 10 but it is exactly representable as 1.4s1.  And in both ANSI Smalltalk and VT_DECIMAL, the numbers 1.4s1, 1.40s2, 1.400s3, and so on are distinguishable.  So the scale *has* to be something over and above the numeric value.  To put it another way, the scale is determined by what the external application WANTS, not what the Smalltalk code HAS. On Wed, 9 Oct 2019 at 09:24, eftomi <[hidden email]> wrote: > > Hi, I'm preparing a method to write a given Pharo's numerical value into an > external variant of type VT_DECIMAL. The purpose of this type is to retain > accuracy, it uses 12 bytes for "mantissa" and two bytes for a sign and a > scale (i. e. the position of decimal point). What would be the best approach > for the conversion, taken all the possible subclasses of class Number, that > is a Float, a Fraction and the Integers, and to keep the accuracy at the > same level? > > For instance, Fraction's numerator and denominator could be directly imputed > into VT_DECIMAL's value and scale if denominator is a power of 10. What to > do in other cases? > > A nice description of VT_DECIMAL is  here > > . > > Best wishes, > Tomaz > > > > -- > Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html>