Number to VT_DECIMAL

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Number to VT_DECIMAL

eftomi
Hi, I'm preparing a method to write a given Pharo's numerical value into an
external variant of type VT_DECIMAL. The purpose of this type is to retain
accuracy, it uses 12 bytes for "mantissa" and two bytes for a sign and a
scale (i. e. the position of decimal point). What would be the best approach
for the conversion, taken all the possible subclasses of class Number, that
is a Float, a Fraction and the Integers, and to keep the accuracy at the
same level?

For instance, Fraction's numerator and denominator could be directly imputed
into VT_DECIMAL's value and scale if denominator is a power of 10. What to
do in other cases?

A nice description of VT_DECIMAL is  here
<https://bytecomb.com/vba-internals-decimal-variables-and-pointers-in-depth/>
.

Best wishes,
Tomaz



--
Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html

Reply | Threaded
Open this post in threaded view
|

Re: Number to VT_DECIMAL

Richard O'Keefe
VT_DECIMAL sounds like a very close match to ScaledDecimal.
Or it would if ScaledDecimal were consistently implemented between Smalltalks.
The intent behind ScaledDecimal in the ANSI Smalltalk standard appears to
have been an (m,p) representation where m and p are Integers standing for
m * 10^p, for interoperation with DECIMAL fields in SQL databases (and of
course with legacy code in COBOL and PL/I). This
seems clear enough: 'Scaled decimal objects provide a precise representation of
decimal fractions with an explicitly specified number of fractional digits.'

That's not what VisualWorks, Squeak, and Pharo actually do.  Instead
they use a (q,p)
representation where q is an *arbitrary* Fraction and p just says how many
decimal places to *print*.  This leads to some very confusing results, and means
that the class *invented* to handle decimal fixed point is no more
helpful to you in
Pharo than Fraction is.  The reason VW gives is that it "seems useful" that
1000s0/7 * 7 should equal 1000s0, but that's actually *not* a useful
property for an
interoperability class.  If I want a Fraction, I know where to find
it, and if I want it
printed to a certain number of decimal places, I know how to do *that* without
needing a complete number class for the purpose.

VisualAge Smalltalk represents a decimal number as an array of 17 bits, which is
a bit more adequate than VT_DECIMAL, and is precisely the IBM mainframe
"packed decimal" format with an extra byte for a scale.

So what should you do?  Basically, a VT_DECIMAL is a pair (m,p,s) where
m between: 0 and: (2 raisedTo: 96) - 1, p between: 0 and: 28, and
s between: 0 and 1.  So

Number
  methods for: 'converting'
    asVtDecimalParts: p "p is the desired scale"
      |m s|
      s := 0.
      m := (self * (10 raisedToInteger: p)) rounded.
      m < 0 ifTrue: [s := 1. m := m negated].
      ^Array with: m with: p with: s

Note that with a Fraction, saying "is the denominator a power of 10,
and if so which?"
won't work, because 7/5 has a denominator that is not a power of 10 but it is
exactly representable as 1.4s1.  And in both ANSI Smalltalk and VT_DECIMAL,
the numbers 1.4s1, 1.40s2, 1.400s3, and so on are distinguishable.  So the scale
*has* to be something over and above the numeric value.  To put it another way,
the scale is determined by what the external application WANTS, not what the
Smalltalk code HAS.

On Wed, 9 Oct 2019 at 09:24, eftomi <[hidden email]> wrote:

>
> Hi, I'm preparing a method to write a given Pharo's numerical value into an
> external variant of type VT_DECIMAL. The purpose of this type is to retain
> accuracy, it uses 12 bytes for "mantissa" and two bytes for a sign and a
> scale (i. e. the position of decimal point). What would be the best approach
> for the conversion, taken all the possible subclasses of class Number, that
> is a Float, a Fraction and the Integers, and to keep the accuracy at the
> same level?
>
> For instance, Fraction's numerator and denominator could be directly imputed
> into VT_DECIMAL's value and scale if denominator is a power of 10. What to
> do in other cases?
>
> A nice description of VT_DECIMAL is  here
> <https://bytecomb.com/vba-internals-decimal-variables-and-pointers-in-depth/>
> .
>
> Best wishes,
> Tomaz
>
>
>
> --
> Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
>

Reply | Threaded
Open this post in threaded view
|

Re: Number to VT_DECIMAL

Richard O'Keefe
Whoops, Visual Age uses 17 BYTES, not 17 BITS.  Slaps own wrist.

On Wed, 9 Oct 2019 at 22:03, Richard O'Keefe <[hidden email]> wrote:

>
> VT_DECIMAL sounds like a very close match to ScaledDecimal.
> Or it would if ScaledDecimal were consistently implemented between Smalltalks.
> The intent behind ScaledDecimal in the ANSI Smalltalk standard appears to
> have been an (m,p) representation where m and p are Integers standing for
> m * 10^p, for interoperation with DECIMAL fields in SQL databases (and of
> course with legacy code in COBOL and PL/I). This
> seems clear enough: 'Scaled decimal objects provide a precise representation of
> decimal fractions with an explicitly specified number of fractional digits.'
>
> That's not what VisualWorks, Squeak, and Pharo actually do.  Instead
> they use a (q,p)
> representation where q is an *arbitrary* Fraction and p just says how many
> decimal places to *print*.  This leads to some very confusing results, and means
> that the class *invented* to handle decimal fixed point is no more
> helpful to you in
> Pharo than Fraction is.  The reason VW gives is that it "seems useful" that
> 1000s0/7 * 7 should equal 1000s0, but that's actually *not* a useful
> property for an
> interoperability class.  If I want a Fraction, I know where to find
> it, and if I want it
> printed to a certain number of decimal places, I know how to do *that* without
> needing a complete number class for the purpose.
>
> VisualAge Smalltalk represents a decimal number as an array of 17 bits, which is
> a bit more adequate than VT_DECIMAL, and is precisely the IBM mainframe
> "packed decimal" format with an extra byte for a scale.
>
> So what should you do?  Basically, a VT_DECIMAL is a pair (m,p,s) where
> m between: 0 and: (2 raisedTo: 96) - 1, p between: 0 and: 28, and
> s between: 0 and 1.  So
>
> Number
>   methods for: 'converting'
>     asVtDecimalParts: p "p is the desired scale"
>       |m s|
>       s := 0.
>       m := (self * (10 raisedToInteger: p)) rounded.
>       m < 0 ifTrue: [s := 1. m := m negated].
>       ^Array with: m with: p with: s
>
> Note that with a Fraction, saying "is the denominator a power of 10,
> and if so which?"
> won't work, because 7/5 has a denominator that is not a power of 10 but it is
> exactly representable as 1.4s1.  And in both ANSI Smalltalk and VT_DECIMAL,
> the numbers 1.4s1, 1.40s2, 1.400s3, and so on are distinguishable.  So the scale
> *has* to be something over and above the numeric value.  To put it another way,
> the scale is determined by what the external application WANTS, not what the
> Smalltalk code HAS.
>
> On Wed, 9 Oct 2019 at 09:24, eftomi <[hidden email]> wrote:
> >
> > Hi, I'm preparing a method to write a given Pharo's numerical value into an
> > external variant of type VT_DECIMAL. The purpose of this type is to retain
> > accuracy, it uses 12 bytes for "mantissa" and two bytes for a sign and a
> > scale (i. e. the position of decimal point). What would be the best approach
> > for the conversion, taken all the possible subclasses of class Number, that
> > is a Float, a Fraction and the Integers, and to keep the accuracy at the
> > same level?
> >
> > For instance, Fraction's numerator and denominator could be directly imputed
> > into VT_DECIMAL's value and scale if denominator is a power of 10. What to
> > do in other cases?
> >
> > A nice description of VT_DECIMAL is  here
> > <https://bytecomb.com/vba-internals-decimal-variables-and-pointers-in-depth/>
> > .
> >
> > Best wishes,
> > Tomaz
> >
> >
> >
> > --
> > Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
> >

Reply | Threaded
Open this post in threaded view
|

Re: Number to VT_DECIMAL

eftomi
Thanks for the explanation and solution - it works flawlessly and efficient
:-)



--
Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html