[ANN] The STON Specification

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

[ANN] The STON Specification

Sven Van Caekenberghe-2
Hi,

Since there can never be enough documentation I finally took some time to write a more formal description of STON as a data format.

  https://github.com/svenvc/ston/blob/master/ston-spec.md

The idea is to let this stabilise a bit and to then update the two other documents describing STON, where necessary:

  https://github.com/svenvc/ston/blob/master/ston-paper.md
  https://ci.inria.fr/pharo-contribution/job/EnterprisePharoBook/lastSuccessfulBuild/artifact/book-result/STON/STON.html

Also, the latest changes in STON have to make their way to the Pharo image as well.

  https://github.com/svenvc/ston

All feedback is welcome.

Sven


--
Sven Van Caekenberghe
Proudly supporting Pharo
http://pharo.org
http://association.pharo.org
http://consortium.pharo.org





Reply | Threaded
Open this post in threaded view
|

Re: [ANN] The STON Specification

David T. Lewis
This is very clear and well written.

Dave

> Hi,
>
> Since there can never be enough documentation I finally took some time to
> write a more formal description of STON as a data format.
>
>   https://github.com/svenvc/ston/blob/master/ston-spec.md
>
> The idea is to let this stabilise a bit and to then update the two other
> documents describing STON, where necessary:
>
>   https://github.com/svenvc/ston/blob/master/ston-paper.md
>   https://ci.inria.fr/pharo-contribution/job/EnterprisePharoBook/lastSuccessfulBuild/artifact/book-result/STON/STON.html
>
> Also, the latest changes in STON have to make their way to the Pharo image
> as well.
>
>   https://github.com/svenvc/ston
>
> All feedback is welcome.
>
> Sven
>
>
> --
> Sven Van Caekenberghe
> Proudly supporting Pharo
> http://pharo.org
> http://association.pharo.org
> http://consortium.pharo.org
>
>
>
>
>



Reply | Threaded
Open this post in threaded view
|

Re: [ANN] The STON Specification

Shaping1
Can STON be extended to handle Doubles and ArbitraryPrecisionFloats?

Shaping

-----Original Message-----
From: Pharo-dev [mailto:[hidden email]] On Behalf Of
David T. Lewis
Sent: Wednesday, October 31, 2018 14:58
To: Pharo Development List <[hidden email]>
Subject: Re: [Pharo-dev] [ANN] The STON Specification

This is very clear and well written.

Dave

> Hi,
>
> Since there can never be enough documentation I finally took some time
> to write a more formal description of STON as a data format.
>
>   https://github.com/svenvc/ston/blob/master/ston-spec.md
>
> The idea is to let this stabilise a bit and to then update the two
> other documents describing STON, where necessary:
>
>   https://github.com/svenvc/ston/blob/master/ston-paper.md
>  
> https://ci.inria.fr/pharo-contribution/job/EnterprisePharoBook/lastSuc
> cessfulBuild/artifact/book-result/STON/STON.html
>
> Also, the latest changes in STON have to make their way to the Pharo
> image as well.
>
>   https://github.com/svenvc/ston
>
> All feedback is welcome.
>
> Sven
>
>
> --
> Sven Van Caekenberghe
> Proudly supporting Pharo
> http://pharo.org
> http://association.pharo.org
> http://consortium.pharo.org
>
>
>
>
>





Reply | Threaded
Open this post in threaded view
|

Re: [ANN] The STON Specification

Sven Van Caekenberghe-2
In reply to this post by David T. Lewis


> On 31 Oct 2018, at 20:58, David T. Lewis <[hidden email]> wrote:
>
> This is very clear and well written.

Thx.

> Dave
>
>> Hi,
>>
>> Since there can never be enough documentation I finally took some time to
>> write a more formal description of STON as a data format.
>>
>>  https://github.com/svenvc/ston/blob/master/ston-spec.md
>>
>> The idea is to let this stabilise a bit and to then update the two other
>> documents describing STON, where necessary:
>>
>>  https://github.com/svenvc/ston/blob/master/ston-paper.md
>>  https://ci.inria.fr/pharo-contribution/job/EnterprisePharoBook/lastSuccessfulBuild/artifact/book-result/STON/STON.html
>>
>> Also, the latest changes in STON have to make their way to the Pharo image
>> as well.
>>
>>  https://github.com/svenvc/ston
>>
>> All feedback is welcome.
>>
>> Sven
>>
>>
>> --
>> Sven Van Caekenberghe
>> Proudly supporting Pharo
>> http://pharo.org
>> http://association.pharo.org
>> http://consortium.pharo.org
>>
>>
>>
>>
>>
>
>
>


Reply | Threaded
Open this post in threaded view
|

Re: [ANN] The STON Specification

Sven Van Caekenberghe-2
In reply to this post by Shaping1


> On 4 Nov 2018, at 00:48, Shaping <[hidden email]> wrote:
>
> Can STON be extended to handle Doubles and ArbitraryPrecisionFloats?

Being born from a language with a rich numerical hierarchy with fluent, mostly transparent conversions, Smalltalk, STON is deliberately a bit vague on precision and range. Integers, fractions and scaled decimals are infinite precision. Float are doubles, but could be higher precision/range: the number reader just does the calculations with the different parts, assuming some floating point number will come out. Each implementation can react based on its own limitations.

> Shaping
>
> -----Original Message-----
> From: Pharo-dev [mailto:[hidden email]] On Behalf Of
> David T. Lewis
> Sent: Wednesday, October 31, 2018 14:58
> To: Pharo Development List <[hidden email]>
> Subject: Re: [Pharo-dev] [ANN] The STON Specification
>
> This is very clear and well written.
>
> Dave
>
>> Hi,
>>
>> Since there can never be enough documentation I finally took some time
>> to write a more formal description of STON as a data format.
>>
>>  https://github.com/svenvc/ston/blob/master/ston-spec.md
>>
>> The idea is to let this stabilise a bit and to then update the two
>> other documents describing STON, where necessary:
>>
>>  https://github.com/svenvc/ston/blob/master/ston-paper.md
>>
>> https://ci.inria.fr/pharo-contribution/job/EnterprisePharoBook/lastSuc
>> cessfulBuild/artifact/book-result/STON/STON.html
>>
>> Also, the latest changes in STON have to make their way to the Pharo
>> image as well.
>>
>>  https://github.com/svenvc/ston
>>
>> All feedback is welcome.
>>
>> Sven
>>
>>
>> --
>> Sven Van Caekenberghe
>> Proudly supporting Pharo
>> http://pharo.org
>> http://association.pharo.org
>> http://consortium.pharo.org
>>
>>
>>
>>
>>
>
>
>
>
>


Reply | Threaded
Open this post in threaded view
|

[ANN] The STON Specification

Shaping1
In reply to this post by David T. Lewis
(Having domain problems recently.  Please excuse this posting if you have
seen it twice.  I've not seen it appear yet on the list.)


Can STON be extended to handle Doubles and ArbitraryPrecisionFloats?

Shaping

-----Original Message-----
From: Pharo-dev [mailto:[hidden email]] On Behalf Of
David T. Lewis
Sent: Wednesday, October 31, 2018 14:58
To: Pharo Development List <[hidden email]>
Subject: Re: [Pharo-dev] [ANN] The STON Specification

This is very clear and well written.

Dave

> Hi,
>
> Since there can never be enough documentation I finally took some time
> to write a more formal description of STON as a data format.
>
>   https://github.com/svenvc/ston/blob/master/ston-spec.md
>
> The idea is to let this stabilise a bit and to then update the two
> other documents describing STON, where necessary:
>
>   https://github.com/svenvc/ston/blob/master/ston-paper.md
>  
> https://ci.inria.fr/pharo-contribution/job/EnterprisePharoBook/lastSuc
> cessfulBuild/artifact/book-result/STON/STON.html
>
> Also, the latest changes in STON have to make their way to the Pharo
> image as well.
>
>   https://github.com/svenvc/ston
>
> All feedback is welcome.
>
> Sven
>
>
> --
> Sven Van Caekenberghe
> Proudly supporting Pharo
> http://pharo.org
> http://association.pharo.org
> http://consortium.pharo.org
>
>
>
>
>





Reply | Threaded
Open this post in threaded view
|

Re: [ANN] The STON Specification

Julien Delplanque-2
Hello,

STON is able to serialise Pharo’s Floats, what do you mean by double ?

Additionally, I recently asked Sven if t would be possible to store ScaledDecimals (I think it implements what you call ArbitraryPrecisionFloats) without loss of precision.

Before, because STON extends JSON, it was storing all kind of numbers either as float or integer.

Now, thanks to Sven, STON stores ScaledDecimals correctly (without loss of precision through serialisation as float, what was done before).

But I do not know if this change is integrated in recent images yet.

Cheers,

Julien

---
Julien Delplanque
Doctorant à l’Université de Lille
Bâtiment B 40, Avenue Halley 59650 Villeneuve d'Ascq
Numéro de téléphone: +333 59 35 86 40

Le 6 nov. 2018 à 06:58, Shaping <[hidden email]> a écrit :

(Having domain problems recently.  Please excuse this posting if you have
seen it twice.  I've not seen it appear yet on the list.)


Can STON be extended to handle Doubles and ArbitraryPrecisionFloats?

Shaping

-----Original Message-----
From: Pharo-dev [[hidden email]] On Behalf Of
David T. Lewis
Sent: Wednesday, October 31, 2018 14:58
To: Pharo Development List <[hidden email]>
Subject: Re: [Pharo-dev] [ANN] The STON Specification

This is very clear and well written.

Dave

Hi,

Since there can never be enough documentation I finally took some time
to write a more formal description of STON as a data format.

 https://github.com/svenvc/ston/blob/master/ston-spec.md

The idea is to let this stabilise a bit and to then update the two
other documents describing STON, where necessary:

 https://github.com/svenvc/ston/blob/master/ston-paper.md

https://ci.inria.fr/pharo-contribution/job/EnterprisePharoBook/lastSuc
cessfulBuild/artifact/book-result/STON/STON.html

Also, the latest changes in STON have to make their way to the Pharo
image as well.

 https://github.com/svenvc/ston

All feedback is welcome.

Sven


--
Sven Van Caekenberghe
Proudly supporting Pharo
http://pharo.org
http://association.pharo.org
http://consortium.pharo.org











Reply | Threaded
Open this post in threaded view
|

Re: [ANN] The STON Specification

Shaping1

STON is able to serialise Pharo’s Floats, what do you mean by double ?

 

Floating-point numbers in IEEE 64-bit format, 14 or 15 significant digits, with a range between =10^307 and 10^307.

 

Additionally, I recently asked Sven if t would be possible to store ScaledDecimals (I think it implements what you call ArbitraryPrecisionFloats) without loss of precision.

 

I’m referring VW’s Double (and SmallDouble in 64-bit engines/images).  APFs are binary representations that can be arbitrarily large, using Integers (LargePositiveIntegers for example) to model the bits of the mantissa.

 

Before, because STON extends JSON, it was storing all kind of numbers either as float or integer.

 

Now, thanks to Sven, STON stores ScaledDecimals correctly (without loss of precision through serialisation as float, what was done before).

 

But I do not know if this change is integrated in recent images yet.

 

…..

 

Number subclass: #Float

                instanceVariableNames: ''

                classVariableNames: 'E Epsilon Halfpi Infinity Ln10 Ln2 MaxVal MaxValLn MinValLogBase2 NaN NegativeInfinity NegativeZero Pi RadiansPerDegree Sqrt2 ThreePi Twopi'

                poolDictionaries: ''

                category: 'Kernel-Numbers'

 

 

My instances represent IEEE-754 floating-point double-precision numbers.  They have about 16 digits of accuracy and their range is between plus and minus 10^307. Some valid examples are:

               

                8.0 13.3 0.3 2.5e6 1.27e-30 1.27e-31 -12.987654e12

….

 

I see that Pharo’s Float is VW’s Double.   So then I just need to be able to serialize APF.

 

….

FIFloatType subclass: #FFIFloat128

                instanceVariableNames: ''

                classVariableNames: ''

                poolDictionaries: ''

                category: 'UnifiedFFI-Types'

 

 

I'm a 128bits (cuadruple precision) float.

It is usually not used, but some compiler modes support it (__float128 in gcc)

 

THIS IS NOT YET SUPPORTED

….

 

The class above is also from the Pharo 7 image.  This is the largest of the c-type FFIFloats.  Any Float classes of this size and larger for the Smalltalk heap on 64-bit Pharo? 

 

 

Cheers,

 

Shaping

 

 

Le 6 nov. 2018 à 06:58, Shaping <[hidden email]> a écrit :

 

(Having domain problems recently.  Please excuse this posting if you have
seen it twice.  I've not seen it appear yet on the list.)


Can STON be extended to handle Doubles and ArbitraryPrecisionFloats?

Shaping

-----Original Message-----
From: Pharo-dev [[hidden email]] On Behalf Of
David T. Lewis
Sent: Wednesday, October 31, 2018 14:58
To: Pharo Development List <[hidden email]>
Subject: Re: [Pharo-dev] [ANN] The STON Specification

This is very clear and well written.

Dave


Hi,

Since there can never be enough documentation I finally took some time
to write a more formal description of STON as a data format.

 https://github.com/svenvc/ston/blob/master/ston-spec.md

The idea is to let this stabilise a bit and to then update the two
other documents describing STON, where necessary:

 https://github.com/svenvc/ston/blob/master/ston-paper.md

https://ci.inria.fr/pharo-contribution/job/EnterprisePharoBook/lastSuc
cessfulBuild/artifact/book-result/STON/STON.html

Also, the latest changes in STON have to make their way to the Pharo
image as well.

 https://github.com/svenvc/ston

All feedback is welcome.

Sven


--
Sven Van Caekenberghe
Proudly supporting Pharo
http://pharo.org
http://association.pharo.org
http://consortium.pharo.org








 

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] The STON Specification

Eliot Miranda-2
Hi Shaping,

    Pharo (& Squeak & Cuis) Float subclass BoxedFloat64 maps exactly to VW's Double. In 64-bit SmallFloat64 maps exactly to SmallDouble.  But I wonder whether there is any issue here.  STON would use the print strings for (PSC) Float / (VW) Double, and so deseerialization on Pharo would automatically produce the right class.  Going in the other direction might need some help.  APF needs support in PSC before one can port, but are representable as suitably-sized word arrays. There is no support for __float128 anywhere in the VM (e.g. not even in the FFI) on PSC as yet.

On Tue, Nov 6, 2018 at 12:56 AM Shaping <[hidden email]> wrote:

STON is able to serialise Pharo’s Floats, what do you mean by double ?

 

Floating-point numbers in IEEE 64-bit format, 14 or 15 significant digits, with a range between =10^307 and 10^307.

 

Additionally, I recently asked Sven if t would be possible to store ScaledDecimals (I think it implements what you call ArbitraryPrecisionFloats) without loss of precision.

 

I’m referring VW’s Double (and SmallDouble in 64-bit engines/images).  APFs are binary representations that can be arbitrarily large, using Integers (LargePositiveIntegers for example) to model the bits of the mantissa.

 

Before, because STON extends JSON, it was storing all kind of numbers either as float or integer.

 

Now, thanks to Sven, STON stores ScaledDecimals correctly (without loss of precision through serialisation as float, what was done before).

 

But I do not know if this change is integrated in recent images yet.

 

…..

 

Number subclass: #Float

                instanceVariableNames: ''

                classVariableNames: 'E Epsilon Halfpi Infinity Ln10 Ln2 MaxVal MaxValLn MinValLogBase2 NaN NegativeInfinity NegativeZero Pi RadiansPerDegree Sqrt2 ThreePi Twopi'

                poolDictionaries: ''

                category: 'Kernel-Numbers'

 

 

My instances represent IEEE-754 floating-point double-precision numbers.  They have about 16 digits of accuracy and their range is between plus and minus 10^307. Some valid examples are:

               

                8.0 13.3 0.3 2.5e6 1.27e-30 1.27e-31 -12.987654e12

….

 

I see that Pharo’s Float is VW’s Double.   So then I just need to be able to serialize APF.

 

….

FIFloatType subclass: #FFIFloat128

                instanceVariableNames: ''

                classVariableNames: ''

                poolDictionaries: ''

                category: 'UnifiedFFI-Types'

 

 

I'm a 128bits (cuadruple precision) float.

It is usually not used, but some compiler modes support it (__float128 in gcc)

 

THIS IS NOT YET SUPPORTED

….

 

The class above is also from the Pharo 7 image.  This is the largest of the c-type FFIFloats.  Any Float classes of this size and larger for the Smalltalk heap on 64-bit Pharo? 

 

 

Cheers,

 

Shaping

 

 

Le 6 nov. 2018 à 06:58, Shaping <[hidden email]> a écrit :

 

(Having domain problems recently.  Please excuse this posting if you have
seen it twice.  I've not seen it appear yet on the list.)


Can STON be extended to handle Doubles and ArbitraryPrecisionFloats?

Shaping

-----Original Message-----
From: Pharo-dev [[hidden email]] On Behalf Of
David T. Lewis
Sent: Wednesday, October 31, 2018 14:58
To: Pharo Development List <[hidden email]>
Subject: Re: [Pharo-dev] [ANN] The STON Specification

This is very clear and well written.

Dave


Hi,

Since there can never be enough documentation I finally took some time
to write a more formal description of STON as a data format.

 https://github.com/svenvc/ston/blob/master/ston-spec.md

The idea is to let this stabilise a bit and to then update the two
other documents describing STON, where necessary:

 https://github.com/svenvc/ston/blob/master/ston-paper.md

https://ci.inria.fr/pharo-contribution/job/EnterprisePharoBook/lastSuc
cessfulBuild/artifact/book-result/STON/STON.html

Also, the latest changes in STON have to make their way to the Pharo
image as well.

 https://github.com/svenvc/ston

All feedback is welcome.

Sven


--
Sven Van Caekenberghe
Proudly supporting Pharo
http://pharo.org
http://association.pharo.org
http://consortium.pharo.org








 



--
_,,,^..^,,,_
best, Eliot