Fraction and ScaledDecimal should be not be serialized as Float in STON format
Locked
Fraction and ScaledDecimal should be not be serialized as Float in STON format
 
|
Hello,
I realised that in the current implementation, when a number is serialised by STON it is either as an integer literal or as a float literal. There is a risk to loose precision, especially if what you serialise is a ScaledDecimal or a fraction. I propose a simple change to fix this: 1. Add STONWriter>>#isInJsonMode method which returns true if the STONWriter wants to write JSON 2. Add:
3. Add:
This change has a main drawback, storing fractions and scaled decimal will result in a huge overhead. Maybe for ScaledDecimal we can use Pharo’s literal (e.g. 1.12s2). For fractions I don’t know what literal could be used? Or maybe the overhead is fine? Cheers, Julien --- Julien Delplanque Doctorant à l’Université de Lille http://juliendelplanque.be/phd.html Equipe Rmod, Inria Bâtiment B 40, Avenue Halley 59650 Villeneuve d'Ascq Numéro de téléphone: +333 59 35 86 40 |
Re: Fraction and ScaledDecimal should be not be serialized as Float in STON format
|
|
Hi Julien,
Good and interesting point. Your summary is correct: STON, inheriting from JSON so to speak, only knowns about integer and float numbers. All other Smalltalk numbers get converted, which results in a loss of type and precision. That might not be a perfect situation, but nobody complained so far. If we would change it, efficiency has to be taken into account: speed/memory efficiency as well as human typing efficiency/complexity. Compatibility, especially between different Smalltalk and other object languages, is also an aspect. Apart from the current approach, the full STON approach could be, for example for 1/3 and 1.5s2 Fraction { #numerator:1, #denominator:3 } ScaledDecimal { #numerator:3, #denominator:2, #scale:2} For some types, STON uses shorter notations, either using positional arguments Point [ 1, 2 ] Fraction [ 1, 3 ] ScaledFraction [ 3, 2, 2 ] Or a string argument, when there is a clear external representation DateAndTime [ '2018-09-18T16:39:10.325129+02:00' ] Fraction [ '1/3' ] ScaledFraction [ '3/2s2' ] Of course, we could also extend the basic number parser and allow the native (stored) notation 1/3 3/2s2 Note how that last one should use the more exact #storeOn: notation, not the #printOn: I'll have to think about this a bit more, as I can really not choose right now. Sven > On 18 Sep 2018, at 10:34, Julien <[hidden email]> wrote: > > Hello, > > I realised that in the current implementation, when a number is serialised by STON it is either as an integer literal or as a float literal. > > There is a risk to loose precision, especially if what you serialise is a ScaledDecimal or a fraction. > > I propose a simple change to fix this: > > 1. Add STONWriter>>#isInJsonMode method which returns true if the STONWriter wants to write JSON > > 2. Add: > > Fraction>>#stonOn: stonWriter > stonWriter isInJsonMode > ifTrue: [ ^ super stonOn: stonWriter ]. > > stonWriter writeObject: self streamMap: [ :dictionary | > dictionary at: #numerator put: numerator. > dictionary at: #denominator put: denominator ] > > 3. Add: > > ScaledDecimal>>#stonOn: stonWriter > stonWriter isInJsonMode > ifTrue: [ ^ super stonOn: stonWriter ]. > > stonWriter writeObject: self streamMap: [ :dictionary | > dictionary at: #numerator put: numerator. > dictionary at: #denominator put: denominator. > dictionary at: #scale put: scale ] > > > This change has a main drawback, storing fractions and scaled decimal will result in a huge overhead. > > Maybe for ScaledDecimal we can use Pharo’s literal (e.g. 1.12s2). > > For fractions I don’t know what literal could be used? > > Or maybe the overhead is fine? > > Cheers, > > Julien > > --- > Julien Delplanque > Doctorant à l’Université de Lille > http://juliendelplanque.be/phd.html > Equipe Rmod, Inria > Bâtiment B 40, Avenue Halley 59650 Villeneuve d'Ascq > Numéro de téléphone: +333 59 35 86 40 > |
Re: Fraction and ScaledDecimal should be not be serialized as Float in STON format
|
|
Hello,
That’s what I guessed to be the reason of this feature. :-)
Tell me if you need help to implement this feature, I’ll be happy to contribute! :-) Cheers, Julien --- Julien Delplanque Doctorant à l’Université de Lille http://juliendelplanque.be/phd.html Equipe Rmod, Inria Bâtiment B 40, Avenue Halley 59650 Villeneuve d'Ascq Numéro de téléphone: +333 59 35 86 40 |
Re: Fraction and ScaledDecimal should be not be serialized as Float in STON format
|
|
In reply to this post by Sven Van Caekenberghe-2
> On 18 Sep 2018, at 16:45, Sven Van Caekenberghe <[hidden email]> wrote: > > Of course, we could also extend the basic number parser and allow the native (stored) notation > > 1/3 > 3/2s2 > > Note how that last one should use the more exact #storeOn: notation, not the #printOn: > > I'll have to think about this a bit more, as I can really not choose right now. I am still not sure if we should do this or not, but here is a POC implementation of the above. It implements the following syntax extension (see the class comment of STON for the rest). number int int denominator int denominator scale int frac int exp int frac exp denominator / digits scale s digits STONWriter>>#writeFraction: fraction jsonMode ifTrue: [ self writeFloat: fraction asFloat ] ifFalse: [ writeStream print: fraction numerator; nextPut: $/; print: fraction denominator ] STONWriter>>#writeScaledDecimal: scaledDecimal jsonMode ifTrue: [ self writeFloat: scaledDecimal asFloat ] ifFalse: [ writeStream print: scaledDecimal numerator; nextPut: $/; print: scaledDecimal denominator; nextPut: $s; print: scaledDecimal scale ] Fraction>>#stonOn: stonWriter stonWriter writeFraction: self ScaledDecimal>>#stonOn: stonWriter stonWriter writeScaledDecimal: self STONReader>>parseNumber | negated number | negated := readStream peekFor: $-. number := self parseNumberInteger. (readStream peekFor: $/) ifTrue: [ number := Fraction numerator: number denominator: self parseNumberInteger. (readStream peekFor: $s) ifTrue: [ number := ScaledDecimal newFromNumber: number scale: self parseNumberInteger ] ] ifFalse: [ (readStream peekFor: $.) ifTrue: [ number := number + self parseNumberFraction ]. ((readStream peekFor: $e) or: [ readStream peekFor: $E ]) ifTrue: [ number := number * self parseNumberExponent ] ]. negated ifTrue: [ number := number negated ]. self consumeWhitespace. ^ number Sven |
Re: Fraction and ScaledDecimal should be not be serialized as Float in STON format
|
|
I decided to add it for real:
https://github.com/svenvc/ston/commit/9c83e3cc2f00cab83e57f2e10a139d6ecef3cb30 add support for Fraction and ScaledDecimal as direct numeric literals (in STON mode, not in JSON mode) with units tests > On 25 Sep 2018, at 14:13, Sven Van Caekenberghe <[hidden email]> wrote: > > > >> On 18 Sep 2018, at 16:45, Sven Van Caekenberghe <[hidden email]> wrote: >> >> Of course, we could also extend the basic number parser and allow the native (stored) notation >> >> 1/3 >> 3/2s2 >> >> Note how that last one should use the more exact #storeOn: notation, not the #printOn: >> >> I'll have to think about this a bit more, as I can really not choose right now. > > I am still not sure if we should do this or not, but here is a POC implementation of the above. > > It implements the following syntax extension (see the class comment of STON for the rest). > > number > int > int denominator > int denominator scale > int frac > int exp > int frac exp > > denominator > / digits > > scale > s digits > > > STONWriter>>#writeFraction: fraction > jsonMode > ifTrue: [ self writeFloat: fraction asFloat ] > ifFalse: [ > writeStream > print: fraction numerator; > nextPut: $/; > print: fraction denominator ] > > STONWriter>>#writeScaledDecimal: scaledDecimal > jsonMode > ifTrue: [ self writeFloat: scaledDecimal asFloat ] > ifFalse: [ > writeStream > print: scaledDecimal numerator; > nextPut: $/; > print: scaledDecimal denominator; > nextPut: $s; > print: scaledDecimal scale ] > > Fraction>>#stonOn: stonWriter > stonWriter writeFraction: self > > ScaledDecimal>>#stonOn: stonWriter > stonWriter writeScaledDecimal: self > > STONReader>>parseNumber > | negated number | > negated := readStream peekFor: $-. > number := self parseNumberInteger. > (readStream peekFor: $/) > ifTrue: [ > number := Fraction numerator: number denominator: self parseNumberInteger. > (readStream peekFor: $s) > ifTrue: [ number := ScaledDecimal newFromNumber: number scale: self parseNumberInteger ] ] > ifFalse: [ > (readStream peekFor: $.) > ifTrue: [ number := number + self parseNumberFraction ]. > ((readStream peekFor: $e) or: [ readStream peekFor: $E ]) > ifTrue: [ number := number * self parseNumberExponent ] ]. > negated > ifTrue: [ number := number negated ]. > self consumeWhitespace. > ^ number > > Sven > |
Re: Fraction and ScaledDecimal should be not be serialized as Float in STON format
|
|
This is cool because indeed scaledDecimal are nice objects :)
On Tue, Oct 9, 2018 at 9:23 PM Sven Van Caekenberghe <[hidden email]> wrote: > > I decided to add it for real: > > https://github.com/svenvc/ston/commit/9c83e3cc2f00cab83e57f2e10a139d6ecef3cb30 > > add support for Fraction and ScaledDecimal as direct numeric literals (in STON mode, not in JSON mode) with units tests > > > On 25 Sep 2018, at 14:13, Sven Van Caekenberghe <[hidden email]> wrote: > > > > > > > >> On 18 Sep 2018, at 16:45, Sven Van Caekenberghe <[hidden email]> wrote: > >> > >> Of course, we could also extend the basic number parser and allow the native (stored) notation > >> > >> 1/3 > >> 3/2s2 > >> > >> Note how that last one should use the more exact #storeOn: notation, not the #printOn: > >> > >> I'll have to think about this a bit more, as I can really not choose right now. > > > > I am still not sure if we should do this or not, but here is a POC implementation of the above. > > > > It implements the following syntax extension (see the class comment of STON for the rest). > > > > number > > int > > int denominator > > int denominator scale > > int frac > > int exp > > int frac exp > > > > denominator > > / digits > > > > scale > > s digits > > > > > > STONWriter>>#writeFraction: fraction > > jsonMode > > ifTrue: [ self writeFloat: fraction asFloat ] > > ifFalse: [ > > writeStream > > print: fraction numerator; > > nextPut: $/; > > print: fraction denominator ] > > > > STONWriter>>#writeScaledDecimal: scaledDecimal > > jsonMode > > ifTrue: [ self writeFloat: scaledDecimal asFloat ] > > ifFalse: [ > > writeStream > > print: scaledDecimal numerator; > > nextPut: $/; > > print: scaledDecimal denominator; > > nextPut: $s; > > print: scaledDecimal scale ] > > > > Fraction>>#stonOn: stonWriter > > stonWriter writeFraction: self > > > > ScaledDecimal>>#stonOn: stonWriter > > stonWriter writeScaledDecimal: self > > > > STONReader>>parseNumber > > | negated number | > > negated := readStream peekFor: $-. > > number := self parseNumberInteger. > > (readStream peekFor: $/) > > ifTrue: [ > > number := Fraction numerator: number denominator: self parseNumberInteger. > > (readStream peekFor: $s) > > ifTrue: [ number := ScaledDecimal newFromNumber: number scale: self parseNumberInteger ] ] > > ifFalse: [ > > (readStream peekFor: $.) > > ifTrue: [ number := number + self parseNumberFraction ]. > > ((readStream peekFor: $e) or: [ readStream peekFor: $E ]) > > ifTrue: [ number := number * self parseNumberExponent ] ]. > > negated > > ifTrue: [ number := number negated ]. > > self consumeWhitespace. > > ^ number > > > > Sven > > > > |
Re: Fraction and ScaledDecimal should be not be serialized as Float in STON format
|
|
In reply to this post by Sven Van Caekenberghe-2
Sorry, I forgot to answer. This change is indeed really nice.
Thank you for it! Cheers, Julien
--- Julien Delplanque Doctorant à l’Université de Lille http://juliendelplanque.be/phd.html Equipe Rmod, Inria Bâtiment B 40, Avenue Halley 59650 Villeneuve d'Ascq Numéro de téléphone: +333 59 35 86 40
|
«
Return to Pharo Smalltalk Developers
|
1 view|%1 views
| Free forum by Nabble | Edit this page |