[vw7.7] Internationalization bug in NumberReader>>readLatinNumberFrom:type?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

[vw7.7] Internationalization bug in NumberReader>>readLatinNumberFrom:type?

Mark Plas

Hi,

 

I'm trying to use

               

                Locale current currencyPolicy reader readFrom: aStream type: FixedPoint

 

to read in a currency value from a stream. There is a problem with it because the method eventually calls NumberReader>>readLatinNumberFrom:type: and this one gives special meaning to the characters 'deqs' it may encounter during parsing. These characters are given the meaning of a 'coercionClass'. 'e' means Floating point exponent character; 'd' and 'q' mean Double exponent character and 's' means FixedPoint exponent character.

 

Does this interpretation belong in this method?  I would think this is more a feature wanted in programming language parsing, but not to be used in Locale-dependent parsing methods?

 

I want to use it to be able for an end user to enter a monetary value followed by a currency symbol, for instance:

 

                10000 euro

 

Parsing this works fine as long as there is a space between 10000 and 'euro'. But if the user enters '10000euro' without a space then the NumberReader first extracts '10000e' as a floating point number and then the rest of the stream contains only 'uro', which leads to incorrect results.

 

Do you think this is a bug?

 

I'm thinking of overriding this method and removing the coercing part. Or is there another way to achieve what I want to do?

 

Thanks,

Mark

 

NumberReader>>readLatinNumberFrom: aStream type: typeClass

 

                | negative whole precision fractional eChar possibleCoercionClass exp coercionClass saved |

                negative := (aStream peekFor: $-)

                                                               ifTrue: [-1]

                                                               ifFalse: [1].

                precision := nil -> nil.

                coercionClass := Integer.

                whole := self getIntegerPartFrom: aStream digits: precision.

                saved := precision value.

                (aStream peekFor: printPolicy decimalPoint)

                               ifTrue:

                                               [coercionClass := Float.

                                               fractional := self getIntegerPartFrom: aStream digits: precision.

                                               (precision value = 0 and: [saved = 0])

                                                               ifTrue: [self error: (#errNoNumber << #dialogs >> 'No number found.')]]

                               ifFalse:

                                               [precision value = 0

                                                               ifTrue: [self error: (#errNoNumber << #dialogs >> 'No number found.')].

                                               fractional := 0.

                                               precision value: 0].

                eChar := aStream peek.

                eChar == nil

                               ifTrue: [possibleCoercionClass := nil]

                               ifFalse:

                                               [possibleCoercionClass := self chooseFloatRepresentationFor: eChar.

                                               possibleCoercionClass == nil ifFalse: [aStream next]].

                exp := nil.

                possibleCoercionClass == nil

                               ifFalse:

                                               [| endOfNumber neg |

                                               coercionClass := possibleCoercionClass.

                                               endOfNumber := aStream position.

                                               neg := aStream peekFor: $-.

                                               (printPolicy isDigit: aStream peek)

                                                               ifTrue:

                                                                              [exp := self getIntegerPartFrom: aStream digits: nil asValue.

                                                                              neg ifTrue: [exp := exp negated]]

                                                               ifFalse: [aStream position: endOfNumber]].

                ^typeClass

                               coerce: fractional / (10 ** precision value) + whole * negative

                               to: coercionClass

                               precision: precision value

                               exponent: exp

                               exponentChar: eChar


_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
Reply | Threaded
Open this post in threaded view
|

Re: [vw7.7] Internationalization bug in NumberReader>>readLatinNumberFrom:type?

Kooyman, Les
Mark:
 
I'm not sure what you illuminate is a bug, that depends as you suggest on what is the appropriate level of function implemented by NumberReader.
 
A lot of things related to internationalization are being exercised seriously in different locales now - we expect to sand a lot of rough edges in the next two releases.
 
I've created AR 60131: Review NumberReader, TimestampReader. It's mission is to review functionality in this area with an eye to the concerns you raised as to appropriateness of locating this coercion in these methods, and to effect a resolution on this and related number parsing issues.  Part of the issue is ambiguity about how much fuzziness these reader methods can handle before figuratively throwing up their hands.
 
Clearing up what you can expect in terms of functionality when reading textual representations of numbers and currencies is high on the list. Textual representations that we create we should be able to read and interpret. Beyond that it gets a bit less clear.
 
In general, the capabilities of TimestampReader have been enhanced for 7.7, but NumberReader has not had similar adjustments, and needs some. The exact nature of what they should be needs to be worked out. If the coercion is moved, where it should reside and effects on existing programming need to be considered.
 
In the meantime, you should deal with the issue locally and expect to reappraise your workaround when the follow-on release after 7.7.1 comes out.
 
Les Kooyman
Cincom Smalltalk


From: [hidden email] on behalf of Mark Plas
Sent: Thu 5/6/2010 12:55 AM
To: vwnc NC
Subject: [vwnc] [vw7.7] Internationalization bug in NumberReader>>readLatinNumberFrom:type?

Hi,

 

I'm trying to use

               

                Locale current currencyPolicy reader readFrom: aStream type: FixedPoint

 

to read in a currency value from a stream. There is a problem with it because the method eventually calls NumberReader>>readLatinNumberFrom:type: and this one gives special meaning to the characters 'deqs' it may encounter during parsing. These characters are given the meaning of a 'coercionClass'. 'e' means Floating point exponent character; 'd' and 'q' mean Double exponent character and 's' means FixedPoint exponent character.

 

Does this interpretation belong in this method?  I would think this is more a feature wanted in programming language parsing, but not to be used in Locale-dependent parsing methods?

 

I want to use it to be able for an end user to enter a monetary value followed by a currency symbol, for instance:

 

                10000 euro

 

Parsing this works fine as long as there is a space between 10000 and 'euro'. But if the user enters '10000euro' without a space then the NumberReader first extracts '10000e' as a floating point number and then the rest of the stream contains only 'uro', which leads to incorrect results.

 

Do you think this is a bug?

 

I'm thinking of overriding this method and removing the coercing part. Or is there another way to achieve what I want to do?

 

Thanks,

Mark

 

NumberReader>>readLatinNumberFrom: aStream type: typeClass

 

                | negative whole precision fractional eChar possibleCoercionClass exp coercionClass saved |

                negative := (aStream peekFor: $-)

                                                               ifTrue: [-1]

                                                               ifFalse: [1].

                precision := nil -> nil.

                coercionClass := Integer.

                whole := self getIntegerPartFrom: aStream digits: precision.

                saved := precision value.

                (aStream peekFor: printPolicy decimalPoint)

                               ifTrue:

                                               [coercionClass := Float.

                                               fractional := self getIntegerPartFrom: aStream digits: precision.

                                               (precision value = 0 and: [saved = 0])

                                                               ifTrue: [self error: (#errNoNumber << #dialogs >> 'No number found.')]]

                               ifFalse:

                                               [precision value = 0

                                                               ifTrue: [self error: (#errNoNumber << #dialogs >> 'No number found.')].

                                               fractional := 0.

                                               precision value: 0].

                eChar := aStream peek.

                eChar == nil

                               ifTrue: [possibleCoercionClass := nil]

                               ifFalse:

                                               [possibleCoercionClass := self chooseFloatRepresentationFor: eChar.

                                               possibleCoercionClass == nil ifFalse: [aStream next]].

                exp := nil.

                possibleCoercionClass == nil

                               ifFalse:

                                               [| endOfNumber neg |

                                               coercionClass := possibleCoercionClass.

                                               endOfNumber := aStream position.

                                               neg := aStream peekFor: $-.

                                               (printPolicy isDigit: aStream peek)

                                                               ifTrue:

                                                                              [exp := self getIntegerPartFrom: aStream digits: nil asValue.

                                                                              neg ifTrue: [exp := exp negated]]

                                                               ifFalse: [aStream position: endOfNumber]].

                ^typeClass

                               coerce: fractional / (10 ** precision value) + whole * negative

                               to: coercionClass

                               precision: precision value

                               exponent: exp

                               exponentChar: eChar


_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
Reply | Threaded
Open this post in threaded view
|

Re: [vw7.7] Internationalization bug in NumberReader>>readLatinNumberFrom:type?

Mark Plas

Thanks Les,

 

I'll make an override then and I can reevaluate when the NumberReader has been reviewed.

 

If you're going to reconsider NumberReader, perhaps one thing you could do is to provide a different way for error handling compared to what happens now? In the current implementation, when something goes wrong an Error exception is thrown:

 

                self error: (#errNoNumber << #dialogs >> 'No number found.')

 

But this is a very general error exception. I think it would be better to throw a different kind of exception so that the handler can be sure that it's a parsing error that occurred instead of some programming error. This exception could contain extra information regarding the position where something went wrong for instance.

 

Best regards,

Mark

 

From: Kooyman, Les [mailto:[hidden email]]
Sent: donderdag 6 mei 2010 18:47
To: Mark Plas; vwnc NC
Subject: RE: [vwnc] [vw7.7] Internationalization bug in NumberReader>>readLatinNumberFrom:type?

 

Mark:

 

I'm not sure what you illuminate is a bug, that depends as you suggest on what is the appropriate level of function implemented by NumberReader.

 

A lot of things related to internationalization are being exercised seriously in different locales now - we expect to sand a lot of rough edges in the next two releases.

 

I've created AR 60131: Review NumberReader, TimestampReader. It's mission is to review functionality in this area with an eye to the concerns you raised as to appropriateness of locating this coercion in these methods, and to effect a resolution on this and related number parsing issues.  Part of the issue is ambiguity about how much fuzziness these reader methods can handle before figuratively throwing up their hands.

 

Clearing up what you can expect in terms of functionality when reading textual representations of numbers and currencies is high on the list. Textual representations that we create we should be able to read and interpret. Beyond that it gets a bit less clear.

 

In general, the capabilities of TimestampReader have been enhanced for 7.7, but NumberReader has not had similar adjustments, and needs some. The exact nature of what they should be needs to be worked out. If the coercion is moved, where it should reside and effects on existing programming need to be considered.

 

In the meantime, you should deal with the issue locally and expect to reappraise your workaround when the follow-on release after 7.7.1 comes out.

 

Les Kooyman

Cincom Smalltalk

 


From: [hidden email] on behalf of Mark Plas
Sent: Thu 5/6/2010 12:55 AM
To: vwnc NC
Subject: [vwnc] [vw7.7] Internationalization bug in NumberReader>>readLatinNumberFrom:type?

Hi,

 

I'm trying to use

               

                Locale current currencyPolicy reader readFrom: aStream type: FixedPoint

 

to read in a currency value from a stream. There is a problem with it because the method eventually calls NumberReader>>readLatinNumberFrom:type: and this one gives special meaning to the characters 'deqs' it may encounter during parsing. These characters are given the meaning of a 'coercionClass'. 'e' means Floating point exponent character; 'd' and 'q' mean Double exponent character and 's' means FixedPoint exponent character.

 

Does this interpretation belong in this method?  I would think this is more a feature wanted in programming language parsing, but not to be used in Locale-dependent parsing methods?

 

I want to use it to be able for an end user to enter a monetary value followed by a currency symbol, for instance:

 

                10000 euro

 

Parsing this works fine as long as there is a space between 10000 and 'euro'. But if the user enters '10000euro' without a space then the NumberReader first extracts '10000e' as a floating point number and then the rest of the stream contains only 'uro', which leads to incorrect results.

 

Do you think this is a bug?

 

I'm thinking of overriding this method and removing the coercing part. Or is there another way to achieve what I want to do?

 

Thanks,

Mark

 

NumberReader>>readLatinNumberFrom: aStream type: typeClass

 

                | negative whole precision fractional eChar possibleCoercionClass exp coercionClass saved |

                negative := (aStream peekFor: $-)

                                                               ifTrue: [-1]

                                                               ifFalse: [1].

                precision := nil -> nil.

                coercionClass := Integer.

                whole := self getIntegerPartFrom: aStream digits: precision.

                saved := precision value.

                (aStream peekFor: printPolicy decimalPoint)

                               ifTrue:

                                               [coercionClass := Float.

                                               fractional := self getIntegerPartFrom: aStream digits: precision.

                                               (precision value = 0 and: [saved = 0])

                                                               ifTrue: [self error: (#errNoNumber << #dialogs >> 'No number found.')]]

                               ifFalse:

                                               [precision value = 0

                                                               ifTrue: [self error: (#errNoNumber << #dialogs >> 'No number found.')].

                                               fractional := 0.

                                               precision value: 0].

                eChar := aStream peek.

                eChar == nil

                               ifTrue: [possibleCoercionClass := nil]

                               ifFalse:

                                               [possibleCoercionClass := self chooseFloatRepresentationFor: eChar.

                                               possibleCoercionClass == nil ifFalse: [aStream next]].

                exp := nil.

                possibleCoercionClass == nil

                               ifFalse:

                                               [| endOfNumber neg |

                                               coercionClass := possibleCoercionClass.

                                               endOfNumber := aStream position.

                                               neg := aStream peekFor: $-.

                                               (printPolicy isDigit: aStream peek)

                                                               ifTrue:

                                                                              [exp := self getIntegerPartFrom: aStream digits: nil asValue.

                                                                              neg ifTrue: [exp := exp negated]]

                                                               ifFalse: [aStream position: endOfNumber]].

                ^typeClass

                               coerce: fractional / (10 ** precision value) + whole * negative

                               to: coercionClass

                               precision: precision value

                               exponent: exp

                               exponentChar: eChar


_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc