D5 - Bug rounding a float (or is it me?)

I have a problem rounding the following numbers. Tested using a
workspace evaluating:

(1/1.1 roundTo: 0.01)  =  0.91   OK

2 - (1/1.1 roundTo: 0.01) = 1.09   OK

BUT,

1 - (1/1.1 roundTo: 0.01) = 9.0e-002   error

I think it should equal 0.09

Regards,
Theo Pronk

Theo,

> 1 - (1/1.1 roundTo: 0.01) = 9.0e-002   error
>
> I think it should equal 0.09

Dolphin uses a Windows function to display float values (see
Float>>printOn:significantFigures:) and this function drops into exponential
format for values below 0.1 (and in various other places as well).  As in
the straightforward

0.09 printString

which still answers '9.0e-002'

To _always_ get a fixed format you normally have to "do it yourself" and
specify the number of decimal places to want to see

n := 1 - (1/1.1 roundTo: 0.01).
n printOn: (s := String writeStream) decimalPlaces: 2.
s contents

I think there's also a bit of the "normal" Float rounding problem in there
somewhere as well - causing

(1 - (1/1.1 roundTo: 0.01)) = 0.09

to answer false.  You just have to be very careful (in all languages) when
using Floats :-(

--
Ian
Due to spamming the reply-to address may only be valid for the next few
days.  Use it to mail me _now_ if you want a longer term contact address.

Theo wrote:

I'm not quite sure what you mean by "OK" vs. "error". If you mean
actually displaying the results of performing the "=", I get "true" for
the first one, and "false" for the other two.

This question or a variation of it keeps coming up. The bottom line is
that Floats are *approximations* and that, as such, checking for
equality of two floating point expressions is generally never a good
idea. Rather, you should subtract the two expressions and see if the
absolute value of the difference is less than an acceptable threshold.
If you really need reproducible equality checks, then Fractions or
perhaps ScaledDecimals, should be used.

(1/1.1 roundTo: 0.01) - 0.91
    display it -> 0.0

2 - (1/1.1 roundTo: 0.01) - 1.09
    display it -> -2.22044604925031e-016

1 - (1/1.1 roundTo: 0.01) - 0.09
    display it -> -2.77555756156289e-017

Close, but not quite equal in the second two cases. You can also use the
method #asTrueFraction to get an idea of what is going on with the
underlying floating point representation.

(1 - (1/1.1 roundTo: 0.01)) asTrueFraction - 0.09 asTrueFraction
    display it -> (-1/36028797018963968)

In other words, the 55th bit is different between the two expressions.

Here is the comment from the asTrueFraction method:
<quote>
Answer a <rational> that precisely represents the binary fractional
value of the receiver using all available bits of the double precision
IEEE floating point representation. Note that because <Float> is an
imprecise representation, the result may have more precision than
appropriate.  For example the decimal number 0.1 cannot be represented
precisely as a binary floating point number, and hence the <Float>
representation is itself only approximate. When <Float> representation
of 0.1 is converted using this method the result is a precisely
equivalent Fraction that is very close to (1/10), but not actually equal
to 0.1."
</quote>

Indeed, you can see for yourself that
0.1 asTrueFraction
    display it -> (3602879701896397/36028797018963968)
is close to (1/10) but not quite.

So it's not a bug. It's just an artifact of using floating point
representation, which is only an approximation of real numbers. And this
is an issue independent of what language is used to specify the floating
point operations. In general, just don't expect two floating point
expressions to be "=".

-------------------------------------------
Bill Dargel            [hidden email]
Shoshana Technologies
100 West Joy Road, Ann Arbor, MI 48105  USA

Hi Ian,
By the look of it I don't really want to use a float at all. Is there a
representation I can use for $s and cents. The calculation I was
actually doing related to determining the amount of tax given a $
expense amount and a tax rate. Eg $1.00 gross tax rate 10% normally
means $0.91 net plus             $0.09 tax.
Regards,
Theo
====================

Ian Bartholomew wrote:

Theo,

> By the look of it I don't really want to use a float at all. Is there a
> representation I can use for $s and cents.

Floats will probably be all right in this situation, you just have to
remember a few rules.

- Don't try to do too many calculations using a single float. The inherent
imprecision in a single calculation is very small but the more calculations
you do on a float the faster the imprecision can get to a level where it
affects the validity of the result.
- Be careful when comparing two values.
- Only do rounding or truncation when you absolutely need to, when you are
displaying a final total for example, and never as part of an ongoing sum.

That, together with your own Float->String conversion that gives the
output precision needed, should be enough.

Some alternatives are -

- A dedicated Currency class that knows about your specific currency. There
was a reference to such a class mentioned in the newsgroup a little while
ago but I don't know what the current situation is. Try Steve Zara's web
site at http://www.serf.org/steve/Dolphin/

- Use the ScaledDecimal class included with Dolphin. This allows you to
specify the precision used - it's basically a Float with constraints.

- Using Floats or Fractions to do the sums and an add-on class to control
the formatting of the results. Coincidentally I just happen to have a goodie
that does that on my web site (http://www.iandb.org.uk)

Like all these things the best method to use just depends on what you are
trying to achieve.

--
Ian
Due to spamming the reply-to address may only be valid for the next few
days.  Use it to mail me _now_ if you want a longer term contact address.

On Sat, 18 May 2002 17:08:56 +0100,
Ian Bartholomew <[hidden email]> wrote:
agreed, but:

> - Only do rounding or truncation when you absolutely need to, when you are
> displaying a final total for example, and never as part of an ongoing sum.

this depends on the situation.

One of my clients uses a MS Access (yuk)
database for generating invoice forms. At first I did as Ian suggested,
and was notified that the running sum was off by a few cents, if you
believed the single items being printed. So I had to make sure that
my running sum was calculated with the same rounded numbers as were
displayed on the form.

So depending on the situation you might want to use integers representing
the smallest currency unit you are interested in.

>
> That, together with your own Float->String conversion that gives the
> output precision needed, should be enough.

Where is the recommended place to put those methods? In one project
I just used Integer>>asCurrency or something like that.

Oh, BTW: Does D5 have a builtin method to format numbers with
localised decimal and thousands separator?

>
> Some alternatives are -

mentioned too late to save me some time :-)

Happy holidays,
stefan

Hi Stefan,

> One of my clients uses a MS Access (yuk)
> database for generating invoice forms. At first I did as Ian suggested,
> and was notified that the running sum was off by a few cents, if you
> believed the single items being printed. So I had to make sure that
> my running sum was calculated with the same rounded numbers as were
> displayed on the form.
>
> So depending on the situation you might want to use integers representing
> the smallest currency unit you are interested in.

Agreed. That might also be the sort of case where a dedicated Currency class
or ScaledDecimal might be a better solution. I was really referring more to
the places where you have a number of internal calculations that result in
one final value but, as you say, it still depends a lot on the situation you
find yourself in.

> Where is the recommended place to put those methods? In one project
> I just used Integer>>asCurrency or something like that.

I would say the best place is in a dedicated CurrencyToText converter class
but in the past I have added methods to Number, added methods to the
application and even done an inline conversion.   Whatever seems easiest at
the time ...

> Oh, BTW: Does D5 have a builtin method to format numbers with
> localised decimal and thousands separator?

Can't say I've noticed anything new and a quick browse in likely places
doesn't throw up anything obvious.

--
Ian
Due to spamming the reply-to address may only be valid for the next few
days.  Use it to mail me _now_ if you want a longer term contact address.