Hi all,
I want to spread an algorithm over several images. For that I need to transfer some objects. For other purposes I already read and write objects via JSON files. To change this to UDP is straightforward. But the conversion to JSON and back takes considerable time, biggest chunk of data is an array with floats. Is there a fast and easy way to transfer objects binary between images? I can transfer only the array of floats (maybe a float array) if that saves time. Thanks, Herbert |
I think ASN1 inside of the Cryptography package may have some speed and
now supports Floats, ScaledDecimals and Fractions, I think. I would love to hear your real world comparison to json, as I am starting to adapt STON to my needs. On 10/20/2017 06:18 AM, Herbert König wrote: > Hi all, > > I want to spread an algorithm over several images. For that I need to > transfer some objects. > > For other purposes I already read and write objects via JSON files. To > change this to UDP is straightforward. But the conversion to JSON and > back takes considerable time, biggest chunk of data is an array with > floats. > > Is there a fast and easy way to transfer objects binary between > images? I can transfer only the array of floats (maybe a float array) > if that saves time. > > > Thanks, > > > Herbert > > > > -- Thank you for your consideration, Alan |
In reply to this post by Herbert König
Yes.
Serialized ReferenceStream .obj files (see attached) You could transfer huge complex objects (complete 6000+ .html pages of swiki) between Squeak and Cuis and with some extra work to Pharo too Or you could convert your JSON to .obj with the attached .mcz and see if more convenient tranfer JSON or .obj yourself On 10/20/17, 07:18, "Herbert König" <[hidden email]> wrote: > Hi all, I want to spread an algorithm over several images. For that I need to > transfer some objects. For other purposes I already read and write objects > via JSON files. To change this to UDP is straightforward. But the conversion > to JSON and back takes considerable time, biggest chunk of data is an array > with floats. Is there a fast and easy way to transfer objects binary between > images? I can transfer only the array of floats (maybe a float array) if that > saves time. Thanks, Herbert |
What about Fuel? |
On Fri, Oct 20, 2017 at 06:33:28PM +0000, Kjell Godo wrote:
> What about Fuel? I thought I had already suggested that, but I guess it was in a different discussion: http://lists.squeakfoundation.org/pipermail/squeak-dev/2017-October/date.html Yes, Fuel is a good choice for serializing between images. ReferenceStream works well, but some things do not work with non-positionable streams, whereas Fuel seems to do a good job with any kind of stream (such as streaming on an OS pipe between two images). The Magma serializer is another candidate. Dave |
In reply to this post by Herbert König
Thanks all,
first I'll try Edgar's way. Nothing new to load/learn and I used ReferenceStream earlier. Cheers, Herbert Am 20.10.2017 um 12:18 schrieb Herbert König: > Hi all, > > I want to spread an algorithm over several images. For that I need to > transfer some objects. > > For other purposes I already read and write objects via JSON files. To > change this to UDP is straightforward. But the conversion to JSON and > back takes considerable time, biggest chunk of data is an array with > floats. > > Is there a fast and easy way to transfer objects binary between > images? I can transfer only the array of floats (maybe a float array) > if that saves time. > > > Thanks, > > > Herbert > > > > |
And noteworthy in addition is that in case you do not have cycles in
your net of objects a DataStream is just fine. http://wiki.squeak.org/squeak/4310 --Hannes On 10/21/17, Herbert König <[hidden email]> wrote: > Thanks all, > > first I'll try Edgar's way. Nothing new to load/learn and I used > ReferenceStream earlier. > > > Cheers, > > > Herbert > > > Am 20.10.2017 um 12:18 schrieb Herbert König: >> Hi all, >> >> I want to spread an algorithm over several images. For that I need to >> transfer some objects. >> >> For other purposes I already read and write objects via JSON files. To >> change this to UDP is straightforward. But the conversion to JSON and >> back takes considerable time, biggest chunk of data is an array with >> floats. >> >> Is there a fast and easy way to transfer objects binary between >> images? I can transfer only the array of floats (maybe a float array) >> if that saves time. >> >> >> Thanks, >> >> >> Herbert >> >> >> >> > > > |
Hi Hannes,
Am Sat, 21 Oct 2017 15:05:19 +0200 schrieb "H. Hirzel" <[hidden email]>: > And noteworthy in addition is that in case you do not have cycles in > your net of objects a DataStream is just fine. > > http://wiki.squeak.org/squeak/4310 Which I don't have so maybe this is even faster because it doesn't employ any logic to deal with cycles. Thanks, Herbert > > --Hannes > > On 10/21/17, Herbert König <[hidden email]> wrote: > > Thanks all, > > > > first I'll try Edgar's way. Nothing new to load/learn and I used > > ReferenceStream earlier. > > > > > > Cheers, > > > > > > Herbert > > > > > > Am 20.10.2017 um 12:18 schrieb Herbert König: > >> Hi all, > >> > >> I want to spread an algorithm over several images. For that I need > >> to transfer some objects. > >> > >> For other purposes I already read and write objects via JSON > >> files. To change this to UDP is straightforward. But the > >> conversion to JSON and back takes considerable time, biggest chunk > >> of data is an array with floats. > >> > >> Is there a fast and easy way to transfer objects binary between > >> images? I can transfer only the array of floats (maybe a float > >> array) if that saves time. > >> > >> > >> Thanks, > >> > >> > >> Herbert > >> > >> > >> > >> > > > > > > > |
There reason I suggested ASN1 is that it is crosee-platofrm, if that is
your requirement. I believe it is effectively binary, for your needs of lots of Floats and I just added support for them too. If you need that... On 10/21/2017 11:38 AM, Herbert König wrote: > Hi Hannes, > > Am Sat, 21 Oct 2017 15:05:19 +0200 > schrieb "H. Hirzel" <[hidden email]>: > >> And noteworthy in addition is that in case you do not have cycles in >> your net of objects a DataStream is just fine. >> >> http://wiki.squeak.org/squeak/4310 > Which I don't have so maybe this is even faster because it doesn't > employ any logic to deal with cycles. > > Thanks, > > Herbert > > >> --Hannes >> >> On 10/21/17, Herbert König <[hidden email]> wrote: >>> Thanks all, >>> >>> first I'll try Edgar's way. Nothing new to load/learn and I used >>> ReferenceStream earlier. >>> >>> >>> Cheers, >>> >>> >>> Herbert >>> >>> >>> Am 20.10.2017 um 12:18 schrieb Herbert König: >>>> Hi all, >>>> >>>> I want to spread an algorithm over several images. For that I need >>>> to transfer some objects. >>>> >>>> For other purposes I already read and write objects via JSON >>>> files. To change this to UDP is straightforward. But the >>>> conversion to JSON and back takes considerable time, biggest chunk >>>> of data is an array with floats. >>>> >>>> Is there a fast and easy way to transfer objects binary between >>>> images? I can transfer only the array of floats (maybe a float >>>> array) if that saves time. >>>> >>>> >>>> Thanks, >>>> >>>> >>>> Herbert >>>> >>>> >>>> >>>> >>> >>> > -- Thank you for your consideration, Alan |
Is JSON the preferred way to transfer data between different Smalltalk platforms? like Dolphin S8 visualWorks GnuSmalltalk Smalltalk/X Amber etc? i would like a way for all the different things to talk to each other why should they all be seperate? I am assuming JSON would be best for going between languages like Smalltalk - Lisp ( it is the easiest to get access to everything has a Package for it ). Are there any Packages that are designed to link different languages together? like Smalltalk - Racket - CommonLisp - SWIProlog etc? i am assuming that sockets + JSON are the easiest quickest way or DCom or what was that Linux d-Bus is d-Bus also on Windows?easy? is COM also on Linux?does anyone use it?hard to use? Dolphin and VisialWork do COM is anyone doing this sort of thing?who would just know?
|
On Sat, Oct 21, 2017 at 04:33:21PM +0000, Kjell Godo wrote:
> Is JSON the preferred way to transfer data between different > Smalltalk platforms? > like Dolphin S8 visualWorks GnuSmalltalk Smalltalk/X Amber etc? > i would like a way for all the different things to talk to each other > why should they all be seperate? > > I am assuming JSON would be best for going between languages like > Smalltalk - Lisp > ( it is the easiest to get access to everything has a Package for it ). > > Are there any Packages that are designed to link different languages > together? > like Smalltalk - Racket - CommonLisp - SWIProlog etc? > i am assuming that sockets + JSON are the easiest quickest way > or DCom or what was that Linux d-Bus is d-Bus also on Windows?easy? > is COM also on Linux?does anyone use it?hard to use? > Dolphin and VisialWork do COM > is anyone doing this sort of thing?who would just know? There is a SOAP implementation by Masashi Umezawa: http://wiki.squeak.org/squeak/1399 You can find it in the SqueakMap package loader (SoapCore, SoapCoreClient, SoapCoreServer, SoapOpera). I believe that the home page is here: http://www.mars.dti.ne.jp/~umejava/smalltalk/soapOpera/soapCore.html I do not know if it still works in current Squeak, or if the package is still being maintained. But I know that it used to work, so it might be worth a look. Dave |
In reply to this post by Kjell Godo
Hi, I used JSON with the addition of carrying the class (JSON being
Javascript born has no concept of classes) because it requires
only very little work after loading the JSON package. (And because
I've been doing some web application where I needed JSON anyway.) Each class needs on the class side: constructFromJson: j and on the instance side: jsonWriteOn: aStream and: constructFromJson: j Simple, stupid and good enough for several of my applications. This should work across dialects if the destination image has the required classes in the same shape. The @Classname designator is a Smalltalk specific extension to JSON. I think it is not good enough to transfer object trees with cyclic dependencies. Other languages with a different ideas of how a class is constructed it might need more work. JSON excerpt: [@IndividualGenom{"halfNum":77,"fitness":2.364011719201257,"fitnesses":null,"geneValues":[-0.7875619210225705,-1.1191086825115324,2.2987547782235476,-2.1996847933965746,3.3254889274350594,-1.7481778033261461
...... Herbert Am 21.10.2017 um 18:33 schrieb Kjell
Godo:
|
In reply to this post by Herbert König
Hi,
DataStream it is. On an Array of objects JSON write avg. 6 s read avg. 1s for 1500 KB ReferenceStream write avg. 1.9 s read avg. 0.4 s for 689 KB DataStream write avg. 1.55 s read avg. 0.13s for 698 KB Cheers, Herbert Am 21.10.2017 um 17:38 schrieb Herbert König: > Hi Hannes, > > Am Sat, 21 Oct 2017 15:05:19 +0200 > schrieb "H. Hirzel" <[hidden email]>: > >> And noteworthy in addition is that in case you do not have cycles in >> your net of objects a DataStream is just fine. >> >> http://wiki.squeak.org/squeak/4310 > Which I don't have so maybe this is even faster because it doesn't > employ any logic to deal with cycles. > > Thanks, > > Herbert > > >> --Hannes >> >> On 10/21/17, Herbert König <[hidden email]> wrote: >>> Thanks all, >>> >>> first I'll try Edgar's way. Nothing new to load/learn and I used >>> ReferenceStream earlier. >>> >>> >>> Cheers, >>> >>> >>> Herbert >>> >>> >>> Am 20.10.2017 um 12:18 schrieb Herbert König: >>>> Hi all, >>>> >>>> I want to spread an algorithm over several images. For that I need >>>> to transfer some objects. >>>> >>>> For other purposes I already read and write objects via JSON >>>> files. To change this to UDP is straightforward. But the >>>> conversion to JSON and back takes considerable time, biggest chunk >>>> of data is an array with floats. >>>> >>>> Is there a fast and easy way to transfer objects binary between >>>> images? I can transfer only the array of floats (maybe a float >>>> array) if that saves time. >>>> >>>> >>>> Thanks, >>>> >>>> >>>> Herbert >>>> >>>> >>>> >>>> >>> >>> > |
By the way, this would be a great library to port for serialization and interfacing to other programming languages:
https://avro.apache.org/docs/current/index.html It gets quite a bit of praise in the book Designing Data-Intensive Applications (https://dataintensive.net) which I can highly recommend. Cheers, Bernhard > Am 22.10.2017 um 17:11 schrieb Herbert König <[hidden email]>: > > Hi, > > DataStream it is. On an Array of objects > > JSON write avg. 6 s read avg. 1s for 1500 KB > > ReferenceStream write avg. 1.9 s read avg. 0.4 s for 689 KB > > DataStream write avg. 1.55 s read avg. 0.13s for 698 KB > > Cheers, > > Herbert > > Am 21.10.2017 um 17:38 schrieb Herbert König: >> Hi Hannes, >> >> Am Sat, 21 Oct 2017 15:05:19 +0200 >> schrieb "H. Hirzel" <[hidden email]>: >> >>> And noteworthy in addition is that in case you do not have cycles in >>> your net of objects a DataStream is just fine. >>> >>> http://wiki.squeak.org/squeak/4310 >> Which I don't have so maybe this is even faster because it doesn't >> employ any logic to deal with cycles. >> >> Thanks, >> >> Herbert >> >> >>> --Hannes >>> >>> On 10/21/17, Herbert König <[hidden email]> wrote: >>>> Thanks all, >>>> >>>> first I'll try Edgar's way. Nothing new to load/learn and I used >>>> ReferenceStream earlier. >>>> >>>> >>>> Cheers, >>>> >>>> >>>> Herbert >>>> >>>> >>>> Am 20.10.2017 um 12:18 schrieb Herbert König: >>>>> Hi all, >>>>> >>>>> I want to spread an algorithm over several images. For that I need >>>>> to transfer some objects. >>>>> >>>>> For other purposes I already read and write objects via JSON >>>>> files. To change this to UDP is straightforward. But the >>>>> conversion to JSON and back takes considerable time, biggest chunk >>>>> of data is an array with floats. >>>>> >>>>> Is there a fast and easy way to transfer objects binary between >>>>> images? I can transfer only the array of floats (maybe a float >>>>> array) if that saves time. >>>>> >>>>> >>>>> Thanks, >>>>> >>>>> >>>>> Herbert >>>>> >>>>> >>>>> >>>>> >>>> >>>> >> > > |
Free forum by Nabble | Edit this page |