rounding errors

> Hi, using formatted number output recently I got the following output:
>
> 1.0 printPaddedWith: $0 to: 2.1. "'01.0'"
> 1.0 printPaddedWith: $0 to: 2.2. "'01.00'"
> 1.0 printPaddedWith: $0 to: 2.3. "'01.00'"    <-- wrong
> 1.0 printPaddedWith: $0 to: 2.4. "'01.000'"   <-- wrong
> 1.0 printPaddedWith: $0 to: 2.5. "'01.00000'"
>
> running Float>>printPaddedWith:to: in the debugger showed me that the reason
> of this odd behaviour seems to be produced by Float>>truncated and/or
> Float>>fractionPart.  I can provoke errors of that kind even easier:
>
> (2.3 fractionPart * 10) truncated. => "2" (instead of 3)
>
> In Primitive 51 (primitiveTruncated for Float>>truncated) I read
>
> frac = modf(rcvr, &trunc);
>
> so the problem looks like an issue with rounding errors in Math-Library's
> modf
> function.
>
> Indeed, I can reproduce the odd result of "(2.3 fractionPart * 10)
> truncated."
> in C++ using modf:
>
> // g++ -lm truncate.cpp && ./a.out
> #include<iostream>
> #include<cmath>
> using namespace std;
> int main(){
>  double f1, f2, f3;
>  modf(10 * 0.3, &f1);
>  cout << f1 << endl;
>  modf(10 * modf(2.3, &f2), &f3);
>  cout << f3 << endl;
> }
> // output:
> // 3 (ok)
> // 2 (error - should be 3)
>
> Question: what can be done about this?
>
> As a matter of fact, several important methods spread over a number of
> classes
> (Float, Number, Morph ... being among them) are senders of Float>>truncated
> or
> Float>>fractionPart, so rounding errors in primitive 51 or 52 could provoke
> unexpected behaviour in surprising places.
>
>
> --
> View this message in context: http://forum.world.st/rounding-errors-tp3327130p3327130.html
> Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
>

> Good catch!
>
> It looks to me like the root cause is that the argument 2.3 is being
> treated as a floating point value, whereas in this context it is really
> intended as a format specifier (i.e. a string, such as you might use
> in a sprintf() function).
>
> I wrote a test to document this bug, and added the fix to Squeak trunk.
> The test and the modified #printPaddedWith:to: method are in the attached
> change set.
>
> Dave
>
>> Hi, using formatted number output recently I got the following output:
>>
>> 1.0 printPaddedWith: $0 to: 2.1. "'01.0'"
>> 1.0 printPaddedWith: $0 to: 2.2. "'01.00'"
>> 1.0 printPaddedWith: $0 to: 2.3. "'01.00'"    <-- wrong
>> 1.0 printPaddedWith: $0 to: 2.4. "'01.000'"   <-- wrong
>> 1.0 printPaddedWith: $0 to: 2.5. "'01.00000'"
>>
>> running Float>>printPaddedWith:to: in the debugger showed me that the reason
>> of this odd behaviour seems to be produced by Float>>truncated and/or
>> Float>>fractionPart.  I can provoke errors of that kind even easier:
>>
>> (2.3 fractionPart * 10) truncated. => "2" (instead of 3)
>>
>> In Primitive 51 (primitiveTruncated for Float>>truncated) I read
>>
>> frac = modf(rcvr, &trunc);
>>
>> so the problem looks like an issue with rounding errors in Math-Library's
>> modf
>> function.
>>
>> Indeed, I can reproduce the odd result of "(2.3 fractionPart * 10)
>> truncated."
>> in C++ using modf:
>>
>> // g++ -lm truncate.cpp && ./a.out
>> #include<iostream>
>> #include<cmath>
>> using namespace std;
>> int main(){
>>  double f1, f2, f3;
>>  modf(10 * 0.3, &f1);
>>  cout << f1 << endl;
>>  modf(10 * modf(2.3, &f2), &f3);
>>  cout << f3 << endl;
>> }
>> // output:
>> // 3 (ok)
>> // 2 (error - should be 3)
>>
>> Question: what can be done about this?
>>
>> As a matter of fact, several important methods spread over a number of
>> classes
>> (Float, Number, Morph ... being among them) are senders of Float>>truncated
>> or
>> Float>>fractionPart, so rounding errors in primitive 51 or 52 could provoke
>> unexpected behaviour in surprising places.
>>
>>
>> --
>> View this message in context: http://forum.world.st/rounding-errors-tp3327130p3327130.html
>> Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
> <Float-printPaddedWith-dtl.1.cs>