In VW7.8 Float nextBefore is broken for powers of two

>
> Let's take a power of two
>
>      e := (Float emin + 1 to: Float emax) atRandom.
>      f := 16r800000 asFloat * (2 raisedTo: e + 1 - Float precision).
>
> We have:
>      f unitOfLeastPrecision = (16r1 asFloat * (2 raisedTo: e + 1 - Float precision)).
>
> With VW7.8 definition of nextBefore, which subtract f - f unitOfLeastPrecision,
> we obtain:
>      f nextBefore = (16r7FFFFF asFloat * (2 raisedTo: e + 1 - Float precision)).
> and once normalized:
>      f nextBefore = (16rFFFFFE asFloat * (2 raisedTo: e - Float precision)).
>
> But there is another Float in between these two,
>      16rFFFFFF asFloat * (2 raisedTo: e - Float precision) < f.
>      16rFFFFFF asFloat * (2 raisedTo: e - Float precision) > f nextBefore.
>
> Yay, we sometimes need to subtract ulp/2...
>
> Nicolas
>
> _______________________________________________
> vwnc mailing list
> [hidden email]
> http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
>