[vwnc] GOODS, TwoByteString and ByteArray

Hello David and list.

The first half is more directed to the helpful people at Cincom, as it
might be a general problem, the second part is more geared towards
David as it's more GOODS-related.

While working with the GOODS client, a typo in the original data
(retrieved from a Windows-based MSSQL database via UnixODBC to my linux
box, don't ask) led me to the following problem:

GOODS sends asByteArray to prepare a string for serializing. If you
happen to pass in a TwoByteString with a sufficiently two-byte
character, the implemenation of this message (located in class String)
fails.

A TwoByteString uses the String implementation of asByteArray to
respond to the message. This is a call to ByteArray>>fromString:
which in turn uses the implementation in its superclass IntegerArray
to happily store a two-byte integer into the byte array.

The following raises an exception:
        (WordArray with: 339) asTwoByteString asByteArray.

However, if you provide an encoding, things work as expected. The following
example does not fail:

        (WordArray with: 339) asTwoByteString asByteArrayEncoding: #utf_8

Maybe TwoByteString should have its own implementation of asByteArray
to provide a suitable byte-based encoding before stuffing the data into
a ByteArray.


The next thing is getting back the "same" (for some unicode-enabled
definition of "same") string that you originally stored. I'm receiving
a ByteArray containing the bytes (duh) of the encoded string, which
I need to convert back using the same encoding that has been used
for storing. The current "asString" in KKStringDescriptor is not smart
enough to handle encoded strings.

As quick fix I'll change the serializing and deserializing to use #utf_8
encoding, but this is something that should probably be taken care of
at the "right" place in the GOODS client. Where would that be?

Later,
s.
       





--
Stefan Schmiedl
EDV-Beratung Schmiedl, Berghangstr. 5, D-93413 Cham
im Büro: 09971 9966 989, am Handy: 0160 9981 6278
_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

> As quick fix I'll change the serializing and deserializing to use #utf_8
> encoding, but this is something that should probably be taken care of
> at the "right" place in the GOODS client. Where would that be?
>  
Yes, using a specific byte encoding such as utf8 is the right thing to
do here. It doesn't make sense to tell a TwoByte or FourByte string to
become a byte array, unless down the track they redefine the 'byte' to
mean 32 bits instead of 8 bits :)

Michael
_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

I think it makes sense for Cincom to change the way VisualWorks handles
floating point values.  Instead of defaulting to single precision when
floats are involved it makes more sense for double precision to be the norm.
It seems odd to me that as a Smalltalk programmer I need to be very
conscious of coercion when it comes to floating point values.

Perhaps there should be a way to let the developer choose a PrecisionPolicy
or some similar thing.

-Carl Gundel
Easy Windows Programming - http://www.libertybasic.com
Easy web programming - http://www.runbasic.com 

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

Coincidentally enough we're fighting through this right now.  GemStone, in their most recent release has pushed the default to FP doubles.  You can't even create small floats in GS anymore, but if, for example, you enter a 0.1 in VW, it generates a small float and that gets pushed to GS.  Unfortunately  a 0.1 small float is not equal to a 0.1 double due to precision.

We're currently trying to figure out the best way to convert all of our small floats to doubles and to ensure no small floats gets introduced.

So I would agree with Carl and either give us an option or just make everything doubles.

-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Carl Gundel
Sent: Sunday, July 27, 2008 8:52 PM
To: [hidden email]
Subject: [vwnc] PrecisionPolicy?

I think it makes sense for Cincom to change the way VisualWorks handles floating point values.  Instead of defaulting to single precision when floats are involved it makes more sense for double precision to be the norm.
It seems odd to me that as a Smalltalk programmer I need to be very conscious of coercion when it comes to floating point values.

Perhaps there should be a way to let the developer choose a PrecisionPolicy or some similar thing.

-Carl Gundel
Easy Windows Programming - http://www.libertybasic.com Easy web programming - http://www.runbasic.com

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc


This message may contain confidential information and is intended for specific recipients unless explicitly noted otherwise. If you have reason to believe you are not an intended recipient of this message, please delete it and notify the sender. This message may not represent the opinion of IntercontinentalExchange, Inc. (ICE), its subsidiaries or affiliates, and does not constitute a contract or guarantee. Unencrypted electronic mail is not secure and the recipient of this message is expected to provide safeguards from viruses and pursue alternate means of communication where privacy or a binding message is desired.


_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

AR 54857...

-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On
Behalf Of Bob Tucker
Sent: Monday, July 28, 2008 5:00 AM
To: [hidden email]
Subject: Re: [vwnc] PrecisionPolicy?

Coincidentally enough we're fighting through this right now.  GemStone,
in their most recent release has pushed the default to FP doubles.  You
can't even create small floats in GS anymore, but if, for example, you
enter a 0.1 in VW, it generates a small float and that gets pushed to
GS.  Unfortunately  a 0.1 small float is not equal to a 0.1 double due
to precision.

We're currently trying to figure out the best way to convert all of our
small floats to doubles and to ensure no small floats gets introduced.

So I would agree with Carl and either give us an option or just make
everything doubles.

-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On
Behalf Of Carl Gundel
Sent: Sunday, July 27, 2008 8:52 PM
To: [hidden email]
Subject: [vwnc] PrecisionPolicy?

I think it makes sense for Cincom to change the way VisualWorks handles
floating point values.  Instead of defaulting to single precision when
floats are involved it makes more sense for double precision to be the
norm.
It seems odd to me that as a Smalltalk programmer I need to be very
conscious of coercion when it comes to floating point values.

Perhaps there should be a way to let the developer choose a
PrecisionPolicy or some similar thing.

-Carl Gundel
Easy Windows Programming - http://www.libertybasic.com Easy web
programming - http://www.runbasic.com

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc


This message may contain confidential information and is intended for
specific recipients unless explicitly noted otherwise. If you have
reason to believe you are not an intended recipient of this message,
please delete it and notify the sender. This message may not represent
the opinion of IntercontinentalExchange, Inc. (ICE), its subsidiaries or
affiliates, and does not constitute a contract or guarantee. Unencrypted
electronic mail is not secure and the recipient of this message is
expected to provide safeguards from viruses and pursue alternate means
of communication where privacy or a binding message is desired.


_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

Bob wrote:
>> We're currently trying to figure out the best way to convert all of our small floats to doubles

Bob and I worked on this. The main thing we were aiming for a double converted from a float to not be dissimilar from a double created from the original number. We needed to fix this:

0.333333 asDouble = 0.333333d
 false

This is the resolution we settled on for VW:

(Double readFrom: 0.333333 printString readStream) = 0.333333d
 true

Here is a comparision of readily available approaches:

0.333333 asFloat
 0.333333
0.333333 asDouble
 0.33333298563957d
0.333333 asRational asDouble
 0.33333298895154d
Double readFrom: 0.333333 printString readStream
 0.333333d

VW floats are implemented with 4-bytes and have six decimals of precision. VW cleverly prints only the float precision that is represented by the 4-bytes. The asDouble example shows that calculations can be performed against floats to return precision that is not intended.

I know the arguments for why this behavior is correct. Don't bother going into those. I also know that floats are a poor substitute for scaled decimals, integers, blah, blah. That is all known. The task we were faced with was not to rewrite the application in a day or two but to come up with a workable solution for a GS upgrade that defaults to 8-byte floats.

Application testing showed we had to convert floats to doubles without changing magnitude. The "asRational asDouble" approach seemed like a perfectly workable solution until we used floats ending with 3. We'd came up with several variations of rounding, but none were ideal because apparently a 4-byte float isn't precisely six decimals of precision as one might hope. The one and only reliable mechanism was VW printString. Yes, I know, nobody would want to rely on #printString for conversion because it is a clear violation of basic coding ideals. It works though. It works perfectly for VW code for all cases we've come across.

VW float conversion was relatively easy using the techniques already shown. GBS also had to be changed to automagically store VW 4-byte floats as GS 8-byte floats. That wasn't bad either. The trickiest part was in creating GS code to do the same kind of conversions. The GemStone recommended technique of just adding zero to a Float or SmallFloat instance to make it a SmallDouble would change the magnitude of a SmallFloat. GS SmallFloat>>asString is not as clever as VW printString because GS shows precision beyond what the implementation supports--it "lies" in the same way float calculations would. Ultimately, the VW code that accurately prints 4-byte floats (without introducing precision) came to the rescue again. Now the GS SmallFloats are also converted by using their string representation.

The end result is that we can now convert floats to GS SmallDouble without changing magnitude--in both VW and GS. Back to the original topic though, you might think that float precision doesn't matter and should be configurable. I like that to be the case; it certainly can be; it is trickier than it looks because increasing precision without changing magnitude requires techniques found in VW #absPrintOn:digits:. Reducing float precision would change magnitude in a potentially detrimental way; many would argue strongly against that. I agree with Carl's point, it is odd that a Smalltalk programmer should be conscious of float coersion. It is nearly as odd as specifying that a number be implemented as a LargeNegativeInteger.

Paul Baumann
[hidden email]


-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Valloud, Andres
Sent: Monday, July 28, 2008 4:23 PM
To: [hidden email]
Subject: Re: [vwnc] PrecisionPolicy?

AR 54857...

-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Bob Tucker
Sent: Monday, July 28, 2008 5:00 AM
To: [hidden email]
Subject: Re: [vwnc] PrecisionPolicy?

Coincidentally enough we're fighting through this right now.  GemStone, in their most recent release has pushed the default to FP doubles.  You can't even create small floats in GS anymore, but if, for example, you enter a 0.1 in VW, it generates a small float and that gets pushed to GS.  Unfortunately  a 0.1 small float is not equal to a 0.1 double due to precision.

We're currently trying to figure out the best way to convert all of our small floats to doubles and to ensure no small floats gets introduced.

So I would agree with Carl and either give us an option or just make everything doubles.

-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Carl Gundel
Sent: Sunday, July 27, 2008 8:52 PM
To: [hidden email]
Subject: [vwnc] PrecisionPolicy?

I think it makes sense for Cincom to change the way VisualWorks handles floating point values.  Instead of defaulting to single precision when floats are involved it makes more sense for double precision to be the norm.
It seems odd to me that as a Smalltalk programmer I need to be very conscious of coercion when it comes to floating point values.

Perhaps there should be a way to let the developer choose a PrecisionPolicy or some similar thing.

-Carl Gundel
Easy Windows Programming - http://www.libertybasic.com Easy web programming - http://www.runbasic.com


This message may contain confidential information and is intended for specific recipients unless explicitly noted otherwise. If you have reason to believe you are not an intended recipient of this message, please delete it and notify the sender. This message may not represent the opinion of IntercontinentalExchange, Inc. (ICE), its subsidiaries or affiliates, and does not constitute a contract or guarantee. Unencrypted electronic mail is not secure and the recipient of this message is expected to provide safeguards from viruses and pursue alternate means of communication where privacy or a binding message is desired.


_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

Paul Baumann wrote:
Paul,


Have you tried replacing "Float" with "Double" in Number
class>>readSmalltalkFloat:from:? This looks like it would replace almost
all uses of 32-bit floats with 64-bit floats in VW. The exception is
when a float is converted from a string that has an explicit 32-bit
exponent specifier (1.0e5). If your code contains, or your users enter,
such strings, you'd also need to change that bit of string conversion code.

This seems like it would be easier (no changes to application code
required, just a recompile to get any literal floats converted) and more
reliable (works as desired for every number, not just the decimal
numbers that can be represented exactly as 32-bit floats).

Regards,

-Martin
_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

Martin,

Interesting. No, I didn't try changing VW to cause it to create doubles instead of floats from read Smalltalk code. It seems like that kind of change could have side effects. Previously compiled Smalltalk code only caused a few dozen 4-byte floats to find their way into GS (at least that we knew about from early tests). Floats created from binary sources would remain floats and then find their way into GS. The way we addressed the float->double conversion worked for all float sources, required no changes to existing code, and didn't change float magnitude just by adding precision for GS.

Interesting idea, thanks for sharing.

--Paul


-----Original Message-----
From: Martin McClure [mailto:[hidden email]]

Paul,

Have you tried replacing "Float" with "Double" in Number
class>>readSmalltalkFloat:from:? This looks like it would replace almost
all uses of 32-bit floats with 64-bit floats in VW. The exception is when a float is converted from a string that has an explicit 32-bit exponent specifier (1.0e5). If your code contains, or your users enter, such strings, you'd also need to change that bit of string conversion code.

This seems like it would be easier (no changes to application code required, just a recompile to get any literal floats converted) and more reliable (works as desired for every number, not just the decimal numbers that can be represented exactly as 32-bit floats).

Regards,

-Martin


This message may contain confidential information and is intended for specific recipients unless explicitly noted otherwise. If you have reason to believe you are not an intended recipient of this message, please delete it and notify the sender. This message may not represent the opinion of IntercontinentalExchange, Inc. (ICE), its subsidiaries or affiliates, and does not constitute a contract or guarantee. Unencrypted electronic mail is not secure and the recipient of this message is expected to provide safeguards from viruses and pursue alternate means of communication where privacy or a binding message is desired.


_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

Paul,

Keep in mind that asRational is not necessarily guaranteed to do what it
claims to do.  While writing the books I also found cases in which code
such as (IIRC) aDouble asRational asDouble can fail.

I'd recommend checking out the construction / deconstruction protocol
for floating point in 7.6.  In particular,

LimitedPrecisionReal class>>fromSignedMantissa:andExponent:


What I'd do is to take a float's signedMantissa, massage it into a
double's signedMantissa (if massaging is necessary), and using the
Double construction protocol to get the exact value you're after.

My 2 cents,

Andres.



Paul Baumann wrote:
_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

Here's an example... at first sight it looks like this should work in
every case... but I have not checked extensively.

(Double fromSignedMantissa: (1.3 signedMantissa bitShift: Double
precision - Float precision) andExponent: 1.3 exponent) asFloat = 1.3 =>
true

... or, even better...

(Double fromSignedMantissa: 1.3 signedMantissa andExponent: 1.3 exponent
+ Double precision - Float precision) asFloat = 1.3 => true

Andres.

-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On
Behalf Of Andres Valloud
Sent: Wednesday, July 30, 2008 12:28 PM
To: [hidden email]
Subject: Re: [vwnc] PrecisionPolicy?

Paul,

Keep in mind that asRational is not necessarily guaranteed to do what it
claims to do.  While writing the books I also found cases in which code
such as (IIRC) aDouble asRational asDouble can fail.

I'd recommend checking out the construction / deconstruction protocol
for floating point in 7.6.  In particular,

LimitedPrecisionReal class>>fromSignedMantissa:andExponent:


What I'd do is to take a float's signedMantissa, massage it into a
double's signedMantissa (if massaging is necessary), and using the
Double construction protocol to get the exact value you're after.

My 2 cents,

Andres.



Paul Baumann wrote:
> Bob wrote:
>  
>>> We're currently trying to figure out the best way to convert all of
>>> our small floats to doubles
>>>      
>
> Bob and I worked on this. The main thing we were aiming for a double
converted from a float to not be dissimilar from a double created from
the original number. We needed to fix this: precision. VW cleverly prints only the float precision that is
represented by the 4-bytes. The asDouble example shows that calculations
can be performed against floats to return precision that is not
intended.
>
> I know the arguments for why this behavior is correct. Don't bother
going into those. I also know that floats are a poor substitute for
scaled decimals, integers, blah, blah. That is all known. The task we
were faced with was not to rewrite the application in a day or two but
to come up with a workable solution for a GS upgrade that defaults to
8-byte floats.
>
> Application testing showed we had to convert floats to doubles without
changing magnitude. The "asRational asDouble" approach seemed like a
perfectly workable solution until we used floats ending with 3. We'd
came up with several variations of rounding, but none were ideal because
apparently a 4-byte float isn't precisely six decimals of precision as
one might hope. The one and only reliable mechanism was VW printString.
Yes, I know, nobody would want to rely on #printString for conversion
because it is a clear violation of basic coding ideals. It works though.
It works perfectly for VW code for all cases we've come across.
>
> VW float conversion was relatively easy using the techniques already
shown. GBS also had to be changed to automagically store VW 4-byte
floats as GS 8-byte floats. That wasn't bad either. The trickiest part
was in creating GS code to do the same kind of conversions. The GemStone
recommended technique of just adding zero to a Float or SmallFloat
instance to make it a SmallDouble would change the magnitude of a
SmallFloat. GS SmallFloat>>asString is not as clever as VW printString
because GS shows precision beyond what the implementation supports--it
"lies" in the same way float calculations would. Ultimately, the VW code
that accurately prints 4-byte floats (without introducing precision)
came to the rescue again. Now the GS SmallFloats are also converted by
using their string representation.
>
> The end result is that we can now convert floats to GS SmallDouble
without changing magnitude--in both VW and GS. Back to the original
topic though, you might think that float precision doesn't matter and
should be configurable. I like that to be the case; it certainly can be;
it is trickier than it looks because increasing precision without
changing magnitude requires techniques found in VW #absPrintOn:digits:.
Reducing float precision would change magnitude in a potentially
detrimental way; many would argue strongly against that. I agree with
Carl's point, it is odd that a Smalltalk programmer should be conscious
of float coersion. It is nearly as odd as specifying that a number be
implemented as a LargeNegativeInteger. GemStone, in their most recent release has pushed the default to FP
doubles.  You can't even create small floats in GS anymore, but if, for
example, you enter a 0.1 in VW, it generates a small float and that gets
pushed to GS.  Unfortunately  a 0.1 small float is not equal to a 0.1
double due to precision.
>
> We're currently trying to figure out the best way to convert all of
our small floats to doubles and to ensure no small floats gets
introduced.
>
> So I would agree with Carl and either give us an option or just make
everything doubles.
>
> -----Original Message-----
> From: [hidden email] [mailto:[hidden email]] On
> Behalf Of Carl Gundel
> Sent: Sunday, July 27, 2008 8:52 PM
> To: [hidden email]
> Subject: [vwnc] PrecisionPolicy?
>
> I think it makes sense for Cincom to change the way VisualWorks
handles floating point values.  Instead of defaulting to single
precision when floats are involved it makes more sense for double
precision to be the norm.
> It seems odd to me that as a Smalltalk programmer I need to be very
conscious of coercion when it comes to floating point values.
>
> Perhaps there should be a way to let the developer choose a
PrecisionPolicy or some similar thing.
>
> -Carl Gundel
> Easy Windows Programming - http://www.libertybasic.com Easy web
> programming - http://www.runbasic.com
>
>
> This message may contain confidential information and is intended for
specific recipients unless explicitly noted otherwise. If you have
reason to believe you are not an intended recipient of this message,
please delete it and notify the sender. This message may not represent
the opinion of IntercontinentalExchange, Inc. (ICE), its subsidiaries or
affiliates, and does not constitute a contract or guarantee. Unencrypted
electronic mail is not secure and the recipient of this message is
expected to provide safeguards from viruses and pursue alternate means
of communication where privacy or a binding message is desired.
>
>
> _______________________________________________
> vwnc mailing list
> [hidden email]
> http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
>
>  

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

Andres,

>> Keep in mind that asRational is not necessarily guaranteed to do what it claims to do.

I understand. I'd said "The 'asRational asDouble' approach seemed like a perfectly workable solution until we used floats ending with 3". 1.23 first showed me that technique was not as precise as it first appeared. I'm sure are many many more.

I'm glad to hear that VW 7.6 has new FP construction protocols. String-based conversion is less than ideal. Fortunately there are very few floats needing conversion in our application--and there will be even fewer over time.

Paul Baumann

-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Andres Valloud
Sent: Wednesday, July 30, 2008 3:28 PM
To: [hidden email]
Subject: Re: [vwnc] PrecisionPolicy?

Paul,

Keep in mind that asRational is not necessarily guaranteed to do what it claims to do.  While writing the books I also found cases in which code such as (IIRC) aDouble asRational asDouble can fail.

I'd recommend checking out the construction / deconstruction protocol for floating point in 7.6.  In particular,

LimitedPrecisionReal class>>fromSignedMantissa:andExponent:


What I'd do is to take a float's signedMantissa, massage it into a double's signedMantissa (if massaging is necessary), and using the Double construction protocol to get the exact value you're after.

My 2 cents,

Andres.



This message may contain confidential information and is intended for specific recipients unless explicitly noted otherwise. If you have reason to believe you are not an intended recipient of this message, please delete it and notify the sender. This message may not represent the opinion of IntercontinentalExchange, Inc. (ICE), its subsidiaries or affiliates, and does not constitute a contract or guarantee. Unencrypted electronic mail is not secure and the recipient of this message is expected to provide safeguards from viruses and pursue alternate means of communication where privacy or a binding message is desired.


_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

Just to clarify, by "failure" I meant UHE.  It's only in a few cases,
but it can happen.

Also note that printString conversion can introduce inaccuracies.  For
example,

(Double pi / 4) sin => 0.70710678118655d

Ok, so then

(Double pi / 4) sin = 0.70710678118655d => false

Fun, isn't it?  Adding digits "fixes" the problem...

(Double pi / 4) sin < 0.70710678118655d true
(Double pi / 4) sin < 0.70710678118654d false
(Double pi / 4) sin < 0.707106781186545d false
(Double pi / 4) sin < 0.707106781186547d false
(Double pi / 4) sin < 0.707106781186549d true
(Double pi / 4) sin < 0.707106781186548d true
(Double pi / 4) sin < 0.7071067811865475d false

Finally,

(Double pi / 4) sin = 0.7071067811865475d true


I do not know if these problems also apply to float to double
conversions, or how, but...

Andres.
 

-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On
Behalf Of Paul Baumann
Sent: Wednesday, July 30, 2008 1:16 PM
To: 'Andres Valloud'; [hidden email]
Subject: Re: [vwnc] PrecisionPolicy?

Andres,

>> Keep in mind that asRational is not necessarily guaranteed to do what
it claims to do.

I understand. I'd said "The 'asRational asDouble' approach seemed like a
perfectly workable solution until we used floats ending with 3". 1.23
first showed me that technique was not as precise as it first appeared.
I'm sure are many many more.

I'm glad to hear that VW 7.6 has new FP construction protocols.
String-based conversion is less than ideal. Fortunately there are very
few floats needing conversion in our application--and there will be even
fewer over time.

Paul Baumann

-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On
Behalf Of Andres Valloud
Sent: Wednesday, July 30, 2008 3:28 PM
To: [hidden email]
Subject: Re: [vwnc] PrecisionPolicy?

Paul,

Keep in mind that asRational is not necessarily guaranteed to do what it
claims to do.  While writing the books I also found cases in which code
such as (IIRC) aDouble asRational asDouble can fail.

I'd recommend checking out the construction / deconstruction protocol
for floating point in 7.6.  In particular,

LimitedPrecisionReal class>>fromSignedMantissa:andExponent:


What I'd do is to take a float's signedMantissa, massage it into a
double's signedMantissa (if massaging is necessary), and using the
Double construction protocol to get the exact value you're after.

My 2 cents,

Andres.



This message may contain confidential information and is intended for
specific recipients unless explicitly noted otherwise. If you have
reason to believe you are not an intended recipient of this message,
please delete it and notify the sender. This message may not represent
the opinion of IntercontinentalExchange, Inc. (ICE), its subsidiaries or
affiliates, and does not constitute a contract or guarantee. Unencrypted
electronic mail is not secure and the recipient of this message is
expected to provide safeguards from viruses and pursue alternate means
of communication where privacy or a binding message is desired.


_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

It's a shame that VW is one of the last Smalltalk implementation not
converting LargePositiveInteger nor Fraction to nearest Floating point
value, be it single or double precision
(unless both numerator and denominator fit in mantissa).
Even a C sscanf does it better, though exact integer/rational arithmetic
is not known as a key feature of the language!

Squeak, Dolphin and gst have incorporated patches for that.
See http://bugs.squeak.org/view.php?id=3564
http://bugs.squeak.org/view.php?id=3568
And there is a solution available at public store...
http://www.cincomsmalltalk.com/publicRepository/SYSBUG-FloatConversion.html
http://www.cincomsmalltalk.com/publicRepository/SYSBUG-FloatConversionTest.html

So when you convert to/from decimal form with printString/readFrom: you
simply verify that you used the same readFrom: that is used in Smalltalk
Scanner/Parser. What you don't verify is that it might be a few bits off.

No use to tell that it would not solve all the problems bound to
rounding errors and inexactness of Floating point...

Even if round to nearest float is solved, past a number of digits, it's
impossible to have a bijective conversion between decimal and binary
Floating point. Some different decimal representations will have a same
binary image in the Floating point set. That's life...


Nicolas


Paul Baumann a écrit :
_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

Nicolas,

The Squeak bug you refer to makes reference to the IEEE rounding rules.  So I pulled out the actual IEEE-754 standard, and in section 5.6 (about binary <--> decimal rounding), the standard says the following:

When the integer M lies outside the range specified in Tables 2 and 3, that is, when M >= 10^9 for single or 10^17 for
double, the implementor may, at his option, alter all significant digits after the ninth for single and seventeenth for
double to other decimal digits, typically 0.

The test case in the Squeak bug says to ensure that

16r1FFFFFFFFFFFF0801 asDouble ~= 16r1FFFFFFFFFFFF0800 asDouble

However, the changed bit is well beyond the 17th decimal digit, and therefore the current VW implementation is within the IEEE standard specification for this particular case.

I suspect that what is being asked for is basically infinite rounding capabilities because some FPUs can use 80 bits internally which then are rounded to 64 bits.  Notwithstanding, this is not what the IEEE standard says.

Andres.


-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of nicolas cellier
Sent: Wednesday, July 30, 2008 3:23 PM
To: [hidden email]
Subject: Re: [vwnc] PrecisionPolicy?

It's a shame that VW is one of the last Smalltalk implementation not converting LargePositiveInteger nor Fraction to nearest Floating point value, be it single or double precision (unless both numerator and denominator fit in mantissa).
Even a C sscanf does it better, though exact integer/rational arithmetic is not known as a key feature of the language!

Squeak, Dolphin and gst have incorporated patches for that.
See http://bugs.squeak.org/view.php?id=3564
http://bugs.squeak.org/view.php?id=3568
And there is a solution available at public store...
http://www.cincomsmalltalk.com/publicRepository/SYSBUG-FloatConversion.html
http://www.cincomsmalltalk.com/publicRepository/SYSBUG-FloatConversionTest.html

So when you convert to/from decimal form with printString/readFrom: you simply verify that you used the same readFrom: that is used in Smalltalk Scanner/Parser. What you don't verify is that it might be a few bits off.

No use to tell that it would not solve all the problems bound to rounding errors and inexactness of Floating point...

Even if round to nearest float is solved, past a number of digits, it's impossible to have a bijective conversion between decimal and binary Floating point. Some different decimal representations will have a same binary image in the Floating point set. That's life...


Nicolas


Paul Baumann a écrit :
_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

Hi, would it better to use a level of of acceptable error when  
comparing floats and doubles?  For example,

( ( ( Double pi/4 ) sin ) - 0.70710678118656d ) < .001

or something like that.

-Conrad

Sent from my iPhone

On Jul 30, 2008, at 2:54 PM, "Valloud, Andres" <[hidden email]>  
wrote
_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

Conrad Taylor wrote:
> Hi, would it better to use a level of of acceptable error when  
> comparing floats and doubles?  For example,
>
> ( ( ( Double pi/4 ) sin ) - 0.70710678118656d ) < .001
>
> or something like that.
>  

Yes, and one can use the message unitOfLeastPrecision to do that, as in:

( ( Double pi/4 ) sin - 0.70710678118656d ) abs < (0.70710678118656d
 unitOfLeastPrecision * tolerance)


Unfortunately, I messed up unitOfLeastPrecision for zero values (doh!).  
Therefore, unitOfLeastPrecision should be amended to read

self = self class zero ifTrue: [^self class fminDenormalized].
^"the original code"


This has been fixed for the next release.

Andres.
_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

Valloud, Andres escreveu:
Which is expected behavior. . . (see f.e.:
http://support.microsoft.com/?scid=kb%3Ben-us%3B42980&x=10&y=9 or for
the gory details: http://docs.sun.com/source/806-3568/ncg_goldberg.html)

The "problem" here is that Float>= is defined as

^aNumber respondsToArithmetic and: [aNumber equalFromFloat: self]

and the sensible would be to test for (aNumber - self) abs <= self epsilon

Where epsilon would a method that returns the minimum representable
quantity for the type of number (like C for example).

Same machinery would need to be put in place for the other selector
though...

my .01999....

--
Cesar Rabak
GNU/Linux User 52247.
Get counted: http://counter.li.org/
_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc