Fun with Number readFrom: What should we do ?

> I totally agree with Diego.
>
> Moreover, I think that each message should do the parsing on exactly one
> format. Something like this:
>
>    (SqNumberParser on: '1.23') number
>    (SqNumberParser on: '1.23e4) scientificNotation; number
>    (SqNumberParser on: '2r1111') base2Notation; number
>    (SqNumberParser on: '') cvsNotation; number "would answer 0"
>
> And then we can add a top level method to try several alternatives (in a
> specific order)
>
>    SqNumberParser>>parse
>        try scientificNotation ifTrue: [^got it]
>        try xxxNotation ifTrue: [^got it]
>        try base2Notation ifTrue: [^got it]
>        try base16Notation ifTrue: [^got it]
>        else ParseException signal
>
> This would be both flexible and generic. Those who know precisely the
> string format of an expected number, can use the specific parse message.
> And if you don't know the string format, just use the generic parse
> message.
>
> String>>sqNumber
>    ^(SqNumberParser on: self) parse; number
>
> By using the Sq prefix, we would avoid clashing with existing methods.
> The code that is relying on the current funny behaviour would remain
> unaffected.
>
> Cheers,
> Francisco
>
>
> ----- Original Message ----- From: "Diego Fernandez" <[hidden email]>
> To: <[hidden email]>; "The general-purpose Squeak developers list"
> <[hidden email]>
> Sent: Thursday, April 27, 2006 2:59 PM
> Subject: Re: Fun with Number readFrom: What should we do ?
>
>
>> My proposition is:
>>
>> Number>>readFrom: aStringOrStream
>>     ^self readFrom: aStringOrStream ifFail: [^self error: 'cannot read a
>> number']
>>
>> If nil is what you want, you will have to write:
>>     value := Number readFrom: aStream ifFail: [nil].
>> A little longer than your proposition:
>>     value := Number readfrom: aStream.
>
> The mine is:
> Number>>readFrom: aStringOrStream
>      ^NumberParser new parse: aStringOrStream
>
> NumberParser>>parse: aStringOrStream
>       "on failure"
>        NumberParsingException signal
>        "or just ParseException"
>
> I think that #readFrom: must be an extension of Number.
> So if you want more control of the parsing you can configure an
> instance of NumberParser.
> (may be you can have a FortranNumberParser :)).
>
>
>

> Hello all,
>
> i noticed that Number readFrom: silently answers 0 when it does not
> encounter some digits:
>
>     Number readFrom: '' readStream.
>     Number readFrom: 'foo-bar' readStream.
>     Number readFrom: '.' readStream.
>
> will all answer 0. Well, i believe there must have been discussions whether
> or not raising an exception in this case, at least recently in vw list. But
> let us say that answering 0 is a feature.

>
>
> Then try:
>
>     Number readFrom: 'rules are made for the others' readStream.
>
> fail with an error: 'Invalid radix'.
> Well, well, don't tell me it is a feature.
>
>
> And just for fun, this one:
>
>     Number readFrom: '.e' readStream.
>
> ah, another feature UndefinedObject doesNotUnderstand: #digitValue.
>
>
> Still want to play ? Try this one:
>
>     Number readFrom: '--1' readStream.
>
> Amazing what we can do with just one readFrom:...
>
>
> Funny, it also accept some un-Smalltalk-ish syntax:
>
>     Number readFrom: '.1e2'.
>     Number readFrom: '1.e1'.
>
> will answer a Float (10.0 like FORTRAN).
> I suppose this one is intentionnaly a feature.
>
>
> I will not analyze here the cases when the stream advance, and the cases
> when it does not... That would be boring...
>
>
> Now i am trying to implement Float readFrom without accumulating rounding
> errors, and this can be the right time for deciding about these cases.
>
> What do you vote for ? exception, answering 0, any other idea ?
> What to do with the stream in case of error ? rewind, keep in place ?
> I'am waiting for your comments.
>
> Nicolas

> The example of the csv file is very badly chosen IMO. CSV files contain
> many different things and applying the same logic you are using for
> numbers here would dictate that String>>readFrom: also answer nil when
> trying to read from an empty string. Moreover, it would dictate that
> Strings use double quotes (if I remember my .csv corretly). This has
> nothing to do with the discussion here - .csv files do NOT contain
> "Smalltalk numbers" so to speak, therefore they need to be separately
> parsed and analyzed and arguing that an empty string should parse as nil
> because .csv files have empty fields is really besides the point of this
> discussion.
>

> I totally agree with Diego.
>
> Moreover, I think that each message should do the parsing on  
> exactly one format. Something like this:
>
>    (SqNumberParser on: '1.23') number
>    (SqNumberParser on: '1.23e4) scientificNotation; number
>    (SqNumberParser on: '2r1111') base2Notation; number
>    (SqNumberParser on: '') cvsNotation; number "would answer 0"
>
> And then we can add a top level method to try several alternatives  
> (in a specific order)
>
>    SqNumberParser>>parse
>        try scientificNotation ifTrue: [^got it]
>        try xxxNotation ifTrue: [^got it]
>        try base2Notation ifTrue: [^got it]
>        try base16Notation ifTrue: [^got it]
>        else ParseException signal
>
> This would be both flexible and generic. Those who know precisely  
> the string format of an expected number, can use the specific parse  
> message. And if you don't know the string format, just use the  
> generic parse message.
>
> String>>sqNumber
>    ^(SqNumberParser on: self) parse; number
>
> By using the Sq prefix, we would avoid clashing with existing  
> methods. The code that is relying on the current funny behaviour  
> would remain unaffected.
>
> Cheers,
> Francisco
>
>
> ----- Original Message ----- From: "Diego Fernandez"  
> <[hidden email]>
> To: <[hidden email]>; "The general-purpose Squeak developers  
> list" <[hidden email]>
> Sent: Thursday, April 27, 2006 2:59 PM
> Subject: Re: Fun with Number readFrom: What should we do ?
>
>
>> My proposition is:
>>
>> Number>>readFrom: aStringOrStream
>>     ^self readFrom: aStringOrStream ifFail: [^self error: 'cannot  
>> read a
>> number']
>>
>> If nil is what you want, you will have to write:
>>     value := Number readFrom: aStream ifFail: [nil].
>> A little longer than your proposition:
>>     value := Number readfrom: aStream.
>
> The mine is:
> Number>>readFrom: aStringOrStream
>      ^NumberParser new parse: aStringOrStream
>
> NumberParser>>parse: aStringOrStream
>       "on failure"
>        NumberParsingException signal
>        "or just ParseException"
>
> I think that #readFrom: must be an extension of Number.
> So if you want more control of the parsing you can configure an
> instance of NumberParser.
> (may be you can have a FortranNumberParser :)).
>
>

> Francisco Garau wrote:
> > This would be both flexible and generic.
>
> And complete and utter overkill in my understanding ;-) Yes, of  
> course, you can do what you just described but by the end of the  
> day, there are two prevalent use cases: 1) I want to read a "Squeak  
> number", e.g., I want to read whatever is considered a valid  
> numeric literal in Squeak. 2) I want to read "something else" and  
> it doesn't matter whether that "something else" is a roman number,  
> a scientific one, a .csv data record. What does matter is that  
> Squeak cannot possibly provide parsers for all numeric formats ever  
> in existence, and neither should it. What it should do is to read  
> its own literals correctly and quickly.
>
> Therefore (coming back to the starting point of this discussion)  
> for Number/Integer/Float>>readFrom: (which by definition are within  
> use case 1) we should not overly generalize reading numbers to the  
> point where it's both slow and gets nothing quite right (since  
> there are almost certainly ambiguous representations). What we  
> should do is to fix what is broken and nothing more; namely that  
> reading a number from an empty string answers zero.
>
> And if you feel that you need more flexibility and format support  
> than Squeak currently provides, feel free to set up a Numeric  
> Parser project on SqueakSource and put that parser there fore  
> anyone to use who needs it. This leaves plenty of room for  
> experimentation (like substituting Number>>readFrom: in the way you  
> are suggesting) and doesn't get into the way of fixing the bug that  
> started the whole discussion.
>
> Cheers,
>   - Andreas
>
> Francisco Garau wrote:
>> I totally agree with Diego.
>> Moreover, I think that each message should do the parsing on  
>> exactly one format. Something like this:
>>    (SqNumberParser on: '1.23') number
>>    (SqNumberParser on: '1.23e4) scientificNotation; number
>>    (SqNumberParser on: '2r1111') base2Notation; number
>>    (SqNumberParser on: '') cvsNotation; number "would answer 0"
>> And then we can add a top level method to try several alternatives  
>> (in a specific order)
>>    SqNumberParser>>parse
>>        try scientificNotation ifTrue: [^got it]
>>        try xxxNotation ifTrue: [^got it]
>>        try base2Notation ifTrue: [^got it]
>>        try base16Notation ifTrue: [^got it]
>>        else ParseException signal
>> This would be both flexible and generic. Those who know precisely  
>> the string format of an expected number, can use the specific  
>> parse message. And if you don't know the string format, just use  
>> the generic parse message.
>> String>>sqNumber
>>    ^(SqNumberParser on: self) parse; number
>> By using the Sq prefix, we would avoid clashing with existing  
>> methods. The code that is relying on the current funny behaviour  
>> would remain unaffected.
>> Cheers,
>> Francisco
>> ----- Original Message ----- From: "Diego Fernandez"  
>> <[hidden email]>
>> To: <[hidden email]>; "The general-purpose Squeak developers  
>> list" <[hidden email]>
>> Sent: Thursday, April 27, 2006 2:59 PM
>> Subject: Re: Fun with Number readFrom: What should we do ?
>>> My proposition is:
>>>
>>> Number>>readFrom: aStringOrStream
>>>     ^self readFrom: aStringOrStream ifFail: [^self error: 'cannot  
>>> read a
>>> number']
>>>
>>> If nil is what you want, you will have to write:
>>>     value := Number readFrom: aStream ifFail: [nil].
>>> A little longer than your proposition:
>>>     value := Number readfrom: aStream.
>> The mine is:
>> Number>>readFrom: aStringOrStream
>>      ^NumberParser new parse: aStringOrStream
>> NumberParser>>parse: aStringOrStream
>>       "on failure"
>>        NumberParsingException signal
>>        "or just ParseException"
>> I think that #readFrom: must be an extension of Number.
>> So if you want more control of the parsing you can configure an
>> instance of NumberParser.
>> (may be you can have a FortranNumberParser :)).
>
>

> Andreas,
>
> I completely agree with what you said except for the first sentence. Why
> do you think it is a "complete and utter overkill"? I can think of three
> possibilities:
>
> 1) The extra class you are adding to the image (SqNumberParser)
> 2) The extra object involved in the parsing
> 3) All the explicit methods that you have to define on the SqNumberParser
> object.
>
> None of them seems an overkill to me. I am obviously missing something.
>
> Cheers,
> Francisco
>

>
> I added the bug report and a few test cases at
> http://bugs.impara.de/view.php?id=3512 last night.
>
> I have a first implementation of SqNumberParser.
> It is more convenient to have a class than a set of messages (see  
> how many are
> necessary in Number>>readFrom:), because we can store some  
> intermediate
> results and better optimize Float reading (number of trailing  
> zeroes is an
> example that can save useless computations). Relax, creating a  
> class does not
> spoil efficiency.

>
> The second advantage of SqNumberParser is that it is a good  
> starting point for
> those wanting their own extension for Number parsing as suggested  
> by Diego
> and Francisco.
> The class could easily have other extensions like indicating the  
> error in the
> TextEditor, like for code...
>
> So far, i suppressed the bugs i know, and the class read Integer  
> ScaledDecimal
> and Float (including nan infinity and negative zero), and pass all the
> NumberParsingTests successfully.
>
> However i still have the bellerophon algorithm (from tony's  
> reference) failing
> with last bit incorrect in a few cases (i can check that with  
> asTrueFraction
> for number greater than 1.0 timestwoPower: 53). So i do not release  
> it yet.
>
> Float>>absPrintOnExactly;base: works rather well and is a good  
> candidate for
> testing readFrom:. However, i do not over trust it, because it  
> includes the
> same code that bugged absPrintOn: (See note in
> http://bugs.impara.de/view.php?id=3493) : it adds 1 to a digit without
> testing if digit=(base-9)... I did not reveal the bug yet.
>
> Nicolas
>
>
>

>
> I have a first implementation of SqNumberParser.
> It is more convenient to have a class than a set of messages (see  
> how many are
> necessary in Number>>readFrom:), because we can store some  
> intermediate
> results and better optimize Float reading (number of trailing  
> zeroes is an
> example that can save useless computations). Relax, creating a  
> class does not
> spoil efficiency.

> The second advantage of SqNumberParser is that it is a good  
> starting point for
> those wanting their own extension for Number parsing as suggested  
> by Diego
> and Francisco.
> The class could easily have other extensions like indicating the  
> error in the
> TextEditor, like for code...
>
> So far, i suppressed the bugs i know, and the class read Integer  
> ScaledDecimal
> and Float (including nan infinity and negative zero), and pass all the
> NumberParsingTests successfully.

> However i still have the bellerophon algorithm (from tony's  
> reference) failing
> with last bit incorrect in a few cases (i can check that with  
> asTrueFraction
> for number greater than 1.0 timestwoPower: 53). So i do not release  
> it yet.
>
> Float>>absPrintOnExactly;base: works rather well and is a good  
> candidate for
> testing readFrom:. However, i do not over trust it, because it  
> includes the
> same code that bugged absPrintOn: (See note in
> http://bugs.impara.de/view.php?id=3493) : it adds 1 to a digit without
> testing if digit=(base-9)... I did not reveal the bug yet.

> Le Vendredi 28 Avril 2006 20:08, nicolas cellier a écrit :
> > However i still have the bellerophon algorithm (from tony's reference)
> > failing with last bit incorrect in a few cases (i can check that with
> > asTrueFraction for number greater than 1.0 timestwoPower: 53). So i do
> > not release it yet.
> >
> > Nicolas
>
> Subjects developped in this (too) long email are:
>  - A) Float readFrom: now round exactly :)
>  - B) my solution use a new ArbitraryPrecisionFloat class :(
>  - C) this class can solve other asFloat problems :)
>  - D) shouldn't the readFrom: fix better be moved to asFloat fix ?
>
> A)-------------------------------------------------------------------------
>---
>
> OK, now i have the Float reading working, also for gradual underflow !
>
> This of course solve Andreas' Numerics question: reading floating point
> constants,
> i now have
> (SqNumberParser on: '6.07710050630396597660e-11') nextNumber hex =
> '3DD0B4611A600000'.
>
> B)-------------------------------------------------------------------------
>---
>
> The solution is not as light as i wanted, because i finally added an
> ArbitraryPrecisionFloat class to handle 64bits arithmetic involved in
> Bellerophon.
> So i do not know if i should publish my solution at
> http://bugs.impara.de/view.php?id=3512, or rather in SM.
> Unfortunately my SM account is blocked inactive...
>
> It's funny how easy creating a new Number class was.
> Of course, ArbitraryPrecisionFloat lacks a short literal form syntax, it
> does not implement usual mathematical functions (default is to convert to
> Float). And it's not optimized at all (please don't compare to hardwired
> Float arithmetic...).
> But it's more general than just reading Float. This is why i wanted to
> publish on SM.
>
> C)-------------------------------------------------------------------------
>---
>
> This can potentially solve other asFloat incorrect rounding behaviour:
>
> C.1) current implementation of LargeInteger>>asFloat.
> will not round to nearest even Float when highBit > 53.
>
> example:
>  16r1FFFFFFFFFFFF081 asFloat hex
>
> The ArbitraryPrecisionFloat implements IEEE rounding mode, so it can cure
> this one easily.
>
> C.2) it is more difficult with Fraction>>asFloat
> which does neither round to nearest even Float...
>
> Bellerophon and ArbitraryPrecisionFloat together can fix the Fraction
> problem (conversion from decimal to binary representation is just a special
> case of these Large fractions).
>
> I now know why i never succedeed in implementing my own Smalltalk Float
> reading in VW, despite of Integer infinite precision... I was over-trusting
> asFloat implementation that suffer the same problem...
>
> D)-------------------------------------------------------------------------
>---
>
> So, maybe we should better move the fixes i wrote from Number>>readFrom: to
> LargeInteger and Fraction>>asFloat,
> Then exactly rounded Float>>readFrom: will be a side effect...
>
> Nicolas

> Andreas,
>
> I completely agree with what you said except for the first sentence. Why
> do you think it is a "complete and utter overkill"? I can think of three
> possibilities:
>
> 1) The extra class you are adding to the image (SqNumberParser)
> 2) The extra object involved in the parsing
> 3) All the explicit methods that you have to define on the SqNumberParser
> object.
>
> None of them seems an overkill to me. I am obviously missing something.
>
> Cheers,
> Francisco
>
>
> This communication is for informational purposes only. It is not intended
> as an offer or solicitation for the purchase or sale of any financial
> instrument or as an official confirmation of any transaction. All market prices,
> data and other information are not warranted as to completeness or accuracy and
> are subject to change without notice. Any comments or statements made herein
> do not necessarily reflect those of JPMorgan Chase & Co., its subsidiaries
> and affiliates.
>
>

> Le Vendredi 28 Avril 2006 20:08, nicolas cellier a écrit :
>> However i still have the bellerophon algorithm (from tony's reference)
>> failing with last bit incorrect in a few cases (i can check that with
>> asTrueFraction for number greater than 1.0 timestwoPower: 53). So i do not
>> release it yet.
>>
>> Nicolas
>
> Subjects developped in this (too) long email are:
>  - A) Float readFrom: now round exactly :)
>  - B) my solution use a new ArbitraryPrecisionFloat class :(
>  - C) this class can solve other asFloat problems :)
>  - D) shouldn't the readFrom: fix better be moved to asFloat fix ?
>
> A)----------------------------------------------------------------------------
>
> OK, now i have the Float reading working, also for gradual underflow !
>
> This of course solve Andreas' Numerics question: reading floating point
> constants,
> i now have
> (SqNumberParser on: '6.07710050630396597660e-11') nextNumber hex =
> '3DD0B4611A600000'.
>
> B)----------------------------------------------------------------------------
>
> The solution is not as light as i wanted, because i finally added an
> ArbitraryPrecisionFloat class to handle 64bits arithmetic involved in
> Bellerophon.
> So i do not know if i should publish my solution at
> http://bugs.impara.de/view.php?id=3512, or rather in SM.
> Unfortunately my SM account is blocked inactive...
>
> It's funny how easy creating a new Number class was.
> Of course, ArbitraryPrecisionFloat lacks a short literal form syntax, it does
> not implement usual mathematical functions (default is to convert to Float).
> And it's not optimized at all (please don't compare to hardwired Float
> arithmetic...).
> But it's more general than just reading Float. This is why i wanted to publish
> on SM.
>
> C)----------------------------------------------------------------------------
>
> This can potentially solve other asFloat incorrect rounding behaviour:
>
> C.1) current implementation of LargeInteger>>asFloat.
> will not round to nearest even Float when highBit > 53.
>
> example:
>  16r1FFFFFFFFFFFF081 asFloat hex
>
> The ArbitraryPrecisionFloat implements IEEE rounding mode, so it can cure this
> one easily.
>
> C.2) it is more difficult with Fraction>>asFloat
> which does neither round to nearest even Float...
>
> Bellerophon and ArbitraryPrecisionFloat together can fix the Fraction problem
> (conversion from decimal to binary representation is just a special case of
> these Large fractions).
>
> I now know why i never succedeed in implementing my own Smalltalk Float
> reading in VW, despite of Integer infinite precision... I was over-trusting
> asFloat implementation that suffer the same problem...
>
> D)----------------------------------------------------------------------------
>
> So, maybe we should better move the fixes i wrote from Number>>readFrom: to
> LargeInteger and Fraction>>asFloat,
> Then exactly rounded Float>>readFrom: will be a side effect...
>
> Nicolas
>
>
>
>