Yes, this kind of thing is inherent in
using binary floating points.
Few decimal fractions can be exactly represented by a binary
float. Ones ending in ".85" can never be exactly represented.
Strings such as "4.85" and "20.85" are rounded to the nearest
representable binary float. Depending on the integer part, the
closest float can be either greater than or less than the ideal
value.
So when further decimal rounding is done, sometimes down is the
correct direction, sometimes up is. Converting the float to a
fraction can show you what the exact value of a float is, and how
it compares to the decimal string.
4.85 asStringUsingFormat: #(1 1 false) = 4.8
4.85 asFraction < (485/100) --> true
20.85 asStringUsingFormat: #(1 1 false) = 20.9
20.85 asFraction > (2085 / 100) --> true
1.85 => 1.9
1.85 asFraction > (185 / 100) --> true
5.85 => 5.8
5.85 asFraction < (585 / 100) --> true
Regards,
-Martin
On 3/4/21 12:32 PM, Marten Feldtmann via Glass wrote:
4.85 asStringUsingFormat: #(1
1 false) = 4.8
20.85 asStringUsingFormat:
#(1 1 false) = 20.9
1.85 => 1.9
5.85 => 5.8
So, actually this is the
typical floating point problem ... :-(
_______________________________________________
Glass mailing list
[hidden email]
https://lists.gemtalksystems.com/mailman/listinfo/glass
_______________________________________________
Glass mailing list
[hidden email]
https://lists.gemtalksystems.com/mailman/listinfo/glass
Locked
