Smalltalk › Squeak › Squeak - Dev

truncatedTo: / roundTo: oddities with fractions

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

7 messages Options

Patrick R.

truncatedTo: / roundTo: oddities with fractions

Hi everyone,

a student recently encountered the following behavior:

fraction := 5 / 9.
(fraction* 100) truncateTo: 0.01. “55.550000000000004"
fraction:= 1/18.
(fraction * 100) roundTo: 0.01. “5.5600000000000005"

This is not what I would expect from #truncateTo: or #roundTo: even when using Floats (especially roundTo:).

Is this what we want or an open issue? I have not found any test covering the protocol.

Bests
Patrick

marcel.taeumel

Re: truncatedTo: / roundTo: oddities with fractions

Hi, all.

I think the issue can be reduced to:

555 * 0.01 "5.55"

5555 * 0.01 "55.550000000000004"

... because #quo: on a Fraction seems to work as expected.

((5/9) * 100 "(500/9)" / 0.01 "5555.555555555556") truncated "5555" * 0.01 "55.550000000000004"

Best,

Marcel

Am 31.07.2019 13:53:49 schrieb Rein, Patrick <[hidden email]>:
Hi everyone,

a student recently encountered the following behavior:

fraction := 5 / 9.
(fraction* 100) truncateTo: 0.01. “55.550000000000004"
fraction:= 1/18.
(fraction * 100) roundTo: 0.01. “5.5600000000000005"

This is not what I would expect from #truncateTo: or #roundTo: even when using Floats (especially roundTo:).

Is this what we want or an open issue? I have not found any test covering the protocol.

Bests
Patrick

David T. Lewis

Re: truncatedTo: / roundTo: oddities with fractions

In reply to this post by Patrick R.

On Wed, Jul 31, 2019 at 11:53:38AM +0000, Rein, Patrick wrote:

> Hi everyone,
>
> a student recently encountered the following behavior:
>
> fraction := 5 / 9.
> (fraction* 100) truncateTo: 0.01. ???55.550000000000004"
> fraction:= 1/18.
> (fraction * 100) roundTo: 0.01. ???5.5600000000000005"
>
> This is not what I would expect from #truncateTo: or #roundTo: even when using Floats (especially roundTo:).
>
> Is this what we want or an open issue? I have not found any test covering the protocol.
>

The results are unexpected, but not wrong.

Why unexpected? When I look at the expression, I intuitively expect
it to perform decimal arithmetic, and it does not do that.

Why not wrong? 0.01 is a float, even though it may have been intended
as an exact decimal number by the writer. Mixing float values in any
arithmetic expression produces an inexact float result, as it should.

To get a result that is both correct and intuitively right, the expression
might better be written like this:

5 / 9 * 100 truncateTo: (1/100) ==> (1111/20)

Dave

Levente Uzonyi

Re: truncatedTo: / roundTo: oddities with fractions

On Wed, 31 Jul 2019, David T. Lewis wrote:

> On Wed, Jul 31, 2019 at 11:53:38AM +0000, Rein, Patrick wrote:
>> Hi everyone,
>>
>> a student recently encountered the following behavior:
>>
>> fraction := 5 / 9.
>> (fraction* 100) truncateTo: 0.01. ???55.550000000000004"
>> fraction:= 1/18.
>> (fraction * 100) roundTo: 0.01. ???5.5600000000000005"
>>
>> This is not what I would expect from #truncateTo: or #roundTo: even when using Floats (especially roundTo:).
>>
>> Is this what we want or an open issue? I have not found any test covering the protocol.
>>
>
> The results are unexpected, but not wrong.
>
> Why unexpected? When I look at the expression, I intuitively expect
> it to perform decimal arithmetic, and it does not do that.
>
> Why not wrong? 0.01 is a float, even though it may have been intended
> as an exact decimal number by the writer. Mixing float values in any
> arithmetic expression produces an inexact float result, as it should.
>
> To get a result that is both correct and intuitively right, the expression
> might better be written like this:
>
> 5 / 9 * 100 truncateTo: (1/100) ==> (1111/20)

I think ScaledDecimal is a better fit, as it gives a decimal result:

5 / 9 * 100 truncateTo: 0.01s "==> 55.55s2"

Levente

>
> Dave

Patrick R.

Re: truncatedTo: / roundTo: oddities with fractions

Thanks! That clarifies things. I agree that it makes sense (although still suprising). To make sure we do not have to discuss that again in the future I will add some test cases for that in the next few days. :)

Bests
Patrick
________________________________________
From: Squeak-dev <[hidden email]> on behalf of Levente Uzonyi <[hidden email]>
Sent: Wednesday, July 31, 2019 3:06:05 PM
To: The general-purpose Squeak developers list
Subject: Re: [squeak-dev] truncatedTo: / roundTo: oddities with fractions

On Wed, 31 Jul 2019, David T. Lewis wrote:

I think ScaledDecimal is a better fit, as it gives a decimal result:

5 / 9 * 100 truncateTo: 0.01s "==> 55.55s2"

Levente

>
> Dave

cbc

Re: truncatedTo: / roundTo: oddities with fractions

In reply to this post by Levente Uzonyi

On Wed, Jul 31, 2019 at 6:06 AM Levente Uzonyi <[hidden email]> wrote:

On Wed, 31 Jul 2019, David T. Lewis wrote:

> On Wed, Jul 31, 2019 at 11:53:38AM +0000, Rein, Patrick wrote:
>> Hi everyone,
>>
>> a student recently encountered the following behavior:
>>
>> fraction := 5 / 9.
>> (fraction* 100) truncateTo: 0.01. ???55.550000000000004"
>> fraction:= 1/18.
>> (fraction * 100) roundTo: 0.01. ???5.5600000000000005"
>>
>> This is not what I would expect from #truncateTo: or #roundTo: even when using Floats (especially roundTo:).
>>
>> Is this what we want or an open issue? I have not found any test covering the protocol.
>>
>
> The results are unexpected, but not wrong.
>
> Why unexpected? When I look at the expression, I intuitively expect
> it to perform decimal arithmetic, and it does not do that.
>
> Why not wrong? 0.01 is a float, even though it may have been intended
> as an exact decimal number by the writer. Mixing float values in any
> arithmetic expression produces an inexact float result, as it should.
>
> To get a result that is both correct and intuitively right, the expression
> might better be written like this:
>
> 5 / 9 * 100 truncateTo: (1/100) ==> (1111/20)

I think ScaledDecimal is a better fit, as it gives a decimal result:

5 / 9 * 100 truncateTo: 0.01s "==> 55.55s2"

To be precise, it still gives you the same result as the fraction result, but it look like you would expect (i.e., it prints as a decimal, properly truncated).

But, yes, probably a better fit.

-cbc

Levente

>
> Dave

Nicolas Cellier

Re: truncatedTo: / roundTo: oddities with fractions

truncateTo: 0.01s or 1/100 is effectively less surprising... but is it good enough?

66.66 truncatesTo: 0.01s as 66.65s.

It might look surprising, bt it's not because 66.66 < (6666/100).

Does it always work?

99.99 < 99.99s.
-> true

so we might expect that it truncates to 99.98.s

99.99 truncateTo: 0.01s
-> 99.99s

Nope...

(0 to: 10000) count: [:i | (i/100) asFloat < (i/100) and: [((i/100) asFloat truncateTo: 0.01s) = (i/100)]].

-> 3720

For the other way (truncate to lower when it should not), I has to go to 5th digit, try this:

(1 to: 100000) select: [:i | (i/100000) asFloat >= (i/100000) and: [((i/100000) asFloat truncateTo: 0.00001s) < (i/100000)]].

The last one is very interesting:

1.0 truncateTo: 0.00001s

-> 0.99999s5

We have 1.0 = 1, so we're dealing with exact decimal fraction operands, and we could expect an exact result...

...but the intermediate 0.00001s asFloat used in (truncateTo:) does not.

One must use (aFloat asFraction truncateTo: 0.00001s) for exactness.

Le jeu. 1 août 2019 à 18:20, Chris Cunningham <[hidden email]> a écrit :

On Wed, Jul 31, 2019 at 6:06 AM Levente Uzonyi <[hidden email]> wrote:
On Wed, 31 Jul 2019, David T. Lewis wrote:

> On Wed, Jul 31, 2019 at 11:53:38AM +0000, Rein, Patrick wrote:
>> Hi everyone,
>>
>> a student recently encountered the following behavior:
>>
>> fraction := 5 / 9.
>> (fraction* 100) truncateTo: 0.01. ???55.550000000000004"
>> fraction:= 1/18.
>> (fraction * 100) roundTo: 0.01. ???5.5600000000000005"
>>
>> This is not what I would expect from #truncateTo: or #roundTo: even when using Floats (especially roundTo:).
>>
>> Is this what we want or an open issue? I have not found any test covering the protocol.
>>
>
> The results are unexpected, but not wrong.
>
> Why unexpected? When I look at the expression, I intuitively expect
> it to perform decimal arithmetic, and it does not do that.
>
> Why not wrong? 0.01 is a float, even though it may have been intended
> as an exact decimal number by the writer. Mixing float values in any
> arithmetic expression produces an inexact float result, as it should.
>
> To get a result that is both correct and intuitively right, the expression
> might better be written like this:
>
> 5 / 9 * 100 truncateTo: (1/100) ==> (1111/20)

I think ScaledDecimal is a better fit, as it gives a decimal result:

5 / 9 * 100 truncateTo: 0.01s "==> 55.55s2"

To be precise, it still gives you the same result as the fraction result, but it look like you would expect (i.e., it prints as a decimal, properly truncated).

But, yes, probably a better fit.

-cbc

Levente

>
> Dave