about mantissa of denormalized floats

In VWNC7.8

Float fminDenormalized mantissa -> 2

By principle of least astonishment, I would expect 1.

All this is due to the definition of exponent which answers the unbiased
exponent of internal representation (-128 is an exceptional value, -127 a
denormalized number, -126 to +127 a normalized one...)

With these definitions, this invariant is OK for all finite Float f:
    (f mantissa * (2 raisedTo: f exponent - f class precision + 1)) = f

But see this:
    Float fminNormalized = (16r800000 * (2.0 raisedTo: -126-23))
But its first denormalized predecessor is:
    (16r7FFFFF * (2.0 raisedTo: -126-23))
Note that you don't have to take -127 as exponent, but rather -126 (Float emin).

In Squeak/Pharo, the invariant is different:
(f significandAsInteger * (2 raisedTo: (f exponent max: f class emin) - f class
precision + 1)) = f

And Squeak exponent is returning f abs floorLog: 2, that is Float
fminDenormalized exponent -> -150, like VW super exponent would (check
LimitedPrecisionReal implementation).

I had to test for an even mantissa, and VW was surprising...

Nicolas

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

Nicolas Cellier <nicolas.cellier.aka.nice <at> gmail.com> writes:

>
> All this is due to the definition of exponent which answers the unbiased
> exponent of internal representation (-128 is an exceptional value, -127 a
> denormalized number, -126 to +127 a normalized one...)
>

Err, exceptional values are +128, hem...



_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

Hello,

On 7/29/2012 4:29 PM, Nicolas Cellier wrote:
> In VWNC7.8
>
> Float fminDenormalized mantissa -> 2
>
> By principle of least astonishment, I would expect 1.

Hmmm, least astonishment according to what?

Part of the problem is that #exponent basically answers the bits in the
exponent field, and so it changes when the floating point values are
denormalized.  Since the exponent changes, then the mantissa *seems* to
shift to the left, but in reality what's happening is that the leading
implicit bit is lost.

So... we *could* change exponent so that it interprets the float and
when the floating point value is denormalized then we answer exponent +
1 etc, but of course

a) that implies breaking backwards compatibility in really subtle ways, and

b) we would take away from the image a really obvious way to tell
whether a floating point value is denormalized.

And what should we do with exponent when the floating point value is +/-
INF, or NaN, etc?...

> In Squeak/Pharo, the invariant is different:
> (f significandAsInteger * (2 raisedTo: (f exponent max: f class emin) - f class
> precision + 1)) = f
>
> And Squeak exponent is returning f abs floorLog: 2, that is Float
> fminDenormalized exponent -> -150, like VW super exponent would (check
> LimitedPrecisionReal implementation).
>
> I had to test for an even mantissa, and VW was surprising...

Well, but again, surprising according to what expectations?  Are you
saying VW was surprising because Squeak/Pharo are different?  Wouldn't
the "problem" here be unreasonable expectations then?

Andres.
_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

Andres Valloud <avalloud <at> cincom.com> writes:

I agree that current exponent implementation is valuable.
So another solution is to keep exponent as is, but change the invariant and let
it be:

(f mantissa * (2 raisedTo: (f exponent max: f class emin) - f class precision +
1)) = f

Then, there will be two ways to tell about gradual underflow,

f exponent < f class emin.
f mantissa highBit < f class precision.

The second way does not discriminate zero and exceptional values, but is IMHO an
important property, see below.

> a) that implies breaking backwards compatibility in really subtle ways, and
>
> b) we would take away from the image a really obvious way to tell
> whether a floating point value is denormalized.
>
> And what should we do with exponent when the floating point value is +/-
> INF, or NaN, etc?...
>
snip...
> >
> > I had to test for an even mantissa, and VW was surprising...
>
> Well, but again, surprising according to what expectations?  Are you
> saying VW was surprising because Squeak/Pharo are different?  Wouldn't
> the "problem" here be unreasonable expectations then?
>
> Andres.
>

No, Squeak/Pharo is not a reference (though I would be glad to see some sort of
standard protocol between dialects), but IEEE 754 is.
Think how intuitive it would be to see leading bits fading as soon as gradual
underflow occurs:

    Float fminNormalized nextBefore mantissa highBit < Float precision.

I'd also like to be able to simply test for an even or odd mantissa, even in
case of underflow. It's required for printing a floating point number
accurately. I can workaround the VW implementation, but thought there could be a
nicer solution.

Nicolas




_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

Hello,

On 7/30/2012 7:16 AM, Nicolas Cellier wrote:

> I agree that current exponent implementation is valuable.

Ok.  I take it you're a bit less surprised now? :)

> So another solution is to keep exponent as is, but change the invariant and let
> it be:
>
> (f mantissa * (2 raisedTo: (f exponent max: f class emin) - f class precision +
> 1)) = f

Where are you getting these invariants from?

> Think how intuitive it would be to see leading bits fading as soon as gradual
> underflow occurs:
>
>      Float fminNormalized nextBefore mantissa highBit < Float precision.

I'm not sure that's supposed to happen... perhaps in the above what's
necessary is a comparison against Float actualPrecision, no?  I mean,
one could have a floating point value with the denormalized exponent and
all mantissa bits set to 1, in which case mantissa highBit is equal to
the precision of the floating point value.  See the difference between
precision and actualPrecision.

> I'd also like to be able to simply test for an even or odd mantissa, even in
> case of underflow. It's required for printing a floating point number
> accurately. I can workaround the VW implementation, but thought there could be a
> nicer solution.

Well, I have about half of the changes needed to make that happen.  I'm
sure you noticed that reading floating point values from a string is not
as robust as it should be.  Are you going to publish your changes somewhere?

Andres.
_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

Andres Valloud <avalloud <at> cincom.com> writes:

If we decompose a Float in sign, mantissa, exponent, there must be some
relationship between the components.

The above one would satisfy both current exponent definition, and my own
mantissa expectations (if we change mantissa accordingly)...

I don't understand this sentence.
Float fminNormalized nextBefore should have bit representation 16r007FFFFF
- it thus has the denormalized exponent (-127 unbiased, or
0 biased)
- and all mantissa bits set, that is the actualPrecision 23 bits only,
because -127 exponent cancels the implied leading bit...

...snip
>
> Well, I have about half of the changes needed to make that happen.  I'm
> sure you noticed that reading floating point values from a string is not
> as robust as it should be.  Are you going to publish your changes somewhere?
>
> Andres.
>

Ah, yes, I failed to have it working 20 years ago because Integer>>asFloat and
Fraction>>asFloat were both liberal (picking a floating point approximation that
was not the nearest one to the exact representation...), and I didn't understood
they were the cause of my problem at that time.

Once above is solved (SYSBUG-FloatConversion in public store), having the
conversion done right is easy (example in SYSEXT-NumberParser in public store).
Having it both fast and right is still difficult though.

Printing is difficult too, but the algorithm is ready in Squeak. I just
published VW port in SYSEXT-NumberPrinter.

Then I can check this:

bad := good := 0.
bad2 := good2 := 0.
0 to: (1 bitShift: 30)-1 do: [:n |
        | f fp |
        f := Float basicNew: 4.
        1 to: 4 do: [:i |
                f basicAt: i put: (n digitAt: i)].
        fp := FloatPrinter scientific alwaysPrintFractionPart: true;  print: f.
        (Core.Number readFrom: fp readStream) = f
                ifTrue: [good := good + 1]
                ifFalse: [bad := bad + 1].
        (VWNumberParser parse: fp readStream) = f
                ifTrue: [good2 := good2 + 1]
                ifFalse: [bad2 := bad2 + 1]].


Nicolas

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

On 7/30/2012 2:43 PM, Nicolas Cellier wrote:
>> Where are you getting these invariants from?
>
> If we decompose a Float in sign, mantissa, exponent, there must be some
> relationship between the components.

I can see that there has to be some relationship because of the
definition of the floating point values themselves.  I was wondering if
you were quoting the invariants from somewhere.

> I don't understand this sentence.
> Float fminNormalized nextBefore should have bit representation 16r007FFFFF

Speaking of that, you might want to look at the packages / parcels
TrippyFloatingPointExtensions (and TrippyIntegerHexExtension is also
helpful sometimes).  With the first one, the "ieee - parsed hex" Trippy
field says

        [0][00][7F FF FF]

> - it thus has the denormalized exponent (-127 unbiased, or
> 0 biased)

0 unbiased, -127 biased... I thought the exponent field was unsigned?

> - and all mantissa bits set, that is the actualPrecision 23 bits only,
> because -127 exponent cancels the implied leading bit...

Oh, sorry, I got confused and thought actualPrecision was 24 and
precision was 23.  Never mind :).

> Ah, yes, I failed to have it working 20 years ago because Integer>>asFloat and
> Fraction>>asFloat were both liberal (picking a floating point approximation that
> was not the nearest one to the exact representation...), and I didn't understood
> they were the cause of my problem at that time.

20 years ago?... that's a long time, what were you doing with floating
point stuff 20 years ago?

In any case, this part I have fixes for.  The code I have parses
integers and fractions to arbitrary precision (i.e., the h value from
the IEEE spec is set to infty) and does the correct rounding as well.
Some of the printing problems are already solved when asRational /
asFraction stop doing approximations and simply return the actual
rational value of the floating point number (that is, mantissa * sign
bit * 2^exponent).  If I remember right there were other problems that I
took care of.  It has been a while since I was working on that, I'll get
going with it again and see what.  Please give me a bit of time to get
back up to speed on this :).

> Once above is solved (SYSBUG-FloatConversion in public store), having the
> conversion done right is easy (example in SYSEXT-NumberParser in public store).
> Having it both fast and right is still difficult though.

I'll look at this code too.

Andres.
_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

Andres Valloud <avalloud <at> cincom.com> writes:


> Speaking of that, you might want to look at the packages / parcels
> TrippyFloatingPointExtensions (and TrippyIntegerHexExtension is also
> helpful sometimes).  With the first one, the "ieee - parsed hex" Trippy
> field says
>
> [0][00][7F FF FF]
>

Nice.


>
> 20 years ago?... that's a long time, what were you doing with floating
> point stuff 20 years ago?
>

A soft like Matlab/Simulink + symbolic description of system by Ordinary
Differential Equations.

Once, a client said he had a problem, saved the case, but the problem vanished
when re-interpreting the floating point values (we abandonned binary storgae
because too hard to maintain).
I said that he should not care of a few ulp errors, and that it was a problem of
numerical stability with his algorithm and data.
But he replied that analyzing stability was his business, and that mine was to
provide a software making problems reproduceable so that he could have a chance
to perform the analysis.
Obviously, the client was right :)

Yes, that's not enough... There is also a problem in:
    (self zero coerce: value) * (10 raisedTo: exponent)
In worse case, it will cumulate 3 inexact float operations, while you can afford
only a final one...

See also
http://stackoverflow.com/questions/85223/how-to-manually-parse-a-floating-point-number-from-a-string/11705617#11705617

Even with a correct Integer rounding, the score of reinterpreting the 10^30
positive finite Float with above code is:

bad:   350222716
good:  723519108

Nicolas



_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

Andres Valloud <avalloud <at> cincom.com> writes:

Nowhere. It's just the natural interpretation of binary representation

   sign * significandInteger * (base raisedTo: exponent)

In http://en.wikipedia.org/wiki/Subnormal_number, you can read

"In binary interchange formats, subnormal numbers are encoded with a biased
exponent of 0, but are interpreted with the value of the smallest allowed
exponent, which is one greater (i.e., as if it were encoded as a 1). In decimal
interchange formats they require no special encoding because the format supports
unnormalized numbers directly."

So [0][00][16r7FFFFF] must be interpreted as

16r7FFFFF * (2 raisedTo: -126 + 1 - 24)

If you let the exponent -127, you have to multiply all mantissa by 2 to
compensate, and end up with mantissa being even for all denormalized Float
    (1 to: (2 raisedTo: Float actualPrecision) - 1) select: [:e | (e * Float
fminDenormalized) mantissa odd].

Hence, exponent of denormalized Float reflect the binary representation, but
mantissa does not. Err... That's what is surprising me.

Also note that (2 raisedTo: -127 + 1 - 24) asFloat will underflow and answer
zero. Fortunately #timesTwoPower: is used instead, and is clever enough to
decompose the operations.

Nicolas


_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

On 7/31/2012 4:52 AM, Nicolas Cellier wrote:

Yeah, I sort of remember this kind of stuff... the new code I have so
far seems impervious to these problems.

I need a bit of time to deal with some makefile changes we're making...
after that I'll try to pick up the floating point stuff and finish it off.

Andres.
_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc