ESUG SummerTalk - Fuel, binary object serializer

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
31 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Re: ESUG SummerTalk - Fuel, binary object serializer

Yoshiki Ohshima-2
At Sat, 28 May 2011 19:26:03 +0200,
Mariano Martinez Peck wrote:
>
> Thanks Yoshiki. So you said Fuel was 2x faster at reading...did you check in writing (serializing) ?
> we would like to know the difference if possible.

  Well, the serializing part is definitely not tuned for performance.
It is even written in OMeta2!  It does not support all bells and
whisles with new kinds of literals and stuff so I cannot really do a
meaningful comparison with Fuel.  But if I compare the serializing
time and deserializing time of my thing, writing is about twice slower
than reading.

-- Yoshiki

_______________________________________________
Esug-list mailing list
[hidden email]
http://lists.esug.org/mailman/listinfo/esug-list_lists.esug.org
Reply | Threaded
Open this post in threaded view
|

Re: ESUG SummerTalk - Fuel, binary object serializer

Mariano Martinez Peck


On Sun, May 29, 2011 at 8:48 PM, Yoshiki Ohshima <[hidden email]> wrote:
At Sat, 28 May 2011 19:26:03 +0200,
Mariano Martinez Peck wrote:
>
> Thanks Yoshiki. So you said Fuel was 2x faster at reading...did you check in writing (serializing) ?
> we would like to know the difference if possible.

 Well, the serializing part is definitely not tuned for performance.
It is even written in OMeta2!  It does not support all bells and
whisles with new kinds of literals and stuff so I cannot really do a
meaningful comparison with Fuel.  

I understand.
 
But if I compare the serializing
time and deserializing time of my thing, writing is about twice slower
than reading.


Yes, I think that is the normal and more or less the expected difference right now.



--
Mariano
http://marianopeck.wordpress.com


_______________________________________________
Esug-list mailing list
[hidden email]
http://lists.esug.org/mailman/listinfo/esug-list_lists.esug.org
Reply | Threaded
Open this post in threaded view
|

Re: ESUG SummerTalk - Fuel, binary object serializer

Mariano Martinez Peck
In reply to this post by Yoshiki Ohshima-2


On Sun, May 29, 2011 at 8:45 PM, Yoshiki Ohshima <[hidden email]> wrote:
At Sat, 28 May 2011 09:33:27 +0200,
Stéphane Ducasse wrote:
>
> Yoshiki
>
> if you want to help testing, improving fuel you are welcome.
> The idea is to make it fast fast fast without vm support.

 Yeah.  The reason for example I went to pad data to 4 bytes

What do you mean by padding data to 4 bytes?  I don't understand :(
 
was that
there may be a clever trick I may be able to do to read data into
arrays "directly" and stuff,

what do you mean by that?  like ImageSegment does?  I mean, include also object headers in the stream and then avoid to recreate objects (using #basicNew) ?
 
but did not get around implementing it.
Next time I can try things, I'll give more close attention to Fuel...


thanks :)
 
-- Yoshiki


_______________________________________________
Esug-list mailing list
[hidden email]
http://lists.esug.org/mailman/listinfo/esug-list_lists.esug.org



--
Mariano
http://marianopeck.wordpress.com


_______________________________________________
Esug-list mailing list
[hidden email]
http://lists.esug.org/mailman/listinfo/esug-list_lists.esug.org
Reply | Threaded
Open this post in threaded view
|

Re: ESUG SummerTalk - Fuel, binary object serializer

Yoshiki Ohshima-2
In reply to this post by Mariano Martinez Peck
At Sun, 29 May 2011 20:51:07 +0200,
Mariano Martinez Peck wrote:
>
>     But if I compare the serializing
>     time and deserializing time of my thing, writing is about twice slower
>     than reading.
>
> Yes, I think that is the normal and more or less the expected difference right now.

  Yes, more or less.  It, however, seems that if I don't use OMeta2,
it can be quite a lot faster.

-- Yoshiki

_______________________________________________
Esug-list mailing list
[hidden email]
http://lists.esug.org/mailman/listinfo/esug-list_lists.esug.org
Reply | Threaded
Open this post in threaded view
|

Re: ESUG SummerTalk - Fuel, binary object serializer

Yoshiki Ohshima-2
In reply to this post by Mariano Martinez Peck
At Sun, 29 May 2011 20:52:45 +0200,
Mariano Martinez Peck wrote:

>
> On Sun, May 29, 2011 at 8:45 PM, Yoshiki Ohshima <[hidden email]> wrote:
>
>     At Sat, 28 May 2011 09:33:27 +0200,
>     Stéphane Ducasse wrote:
>     >
>     > Yoshiki
>     >
>     > if you want to help testing, improving fuel you are welcome.
>     > The idea is to make it fast fast fast without vm support.
>    
>      Yeah.  The reason for example I went to pad data to 4 bytes
>
> What do you mean by padding data to 4 bytes?  I don't understand :(

  Say a string has length of 5 ('abcde'), the data in file for it
would be "97 98 99 100 101 0 0 0".

>     was that
>     there may be a clever trick I may be able to do to read data into
>     arrays "directly" and stuff,
>
> what do you mean by that?  like ImageSegment does?  I mean, include also object headers in the stream and then avoid to recreate objects (using #basicNew) ?

  Not like ImageSegment, but more like #hackBits: reading from file
into a string object of right size.  For pointer arrays and actual
literal fields, I am not sure I can apply this however.

  I would not put object headers, as you would agree^^;

-- Yoshiki

_______________________________________________
Esug-list mailing list
[hidden email]
http://lists.esug.org/mailman/listinfo/esug-list_lists.esug.org
Reply | Threaded
Open this post in threaded view
|

Re: ESUG SummerTalk - Fuel, binary object serializer

Paolo Bonzini-2
In reply to this post by Mariano Martinez Peck
On 05/29/2011 08:51 PM, Mariano Martinez Peck wrote:
>     But if I compare the serializing
>     time and deserializing time of my thing, writing is about twice slower
>     than reading.
>
>
> Yes, I think that is the normal and more or less the expected difference
> right now.
>

Do you want to make some comparisons with the GNU Smalltalk
ObjectDumper?  I can help, or I can even make some comparisons myself if
you give me some test Fuel code and an image to test with.

Paolo

_______________________________________________
Esug-list mailing list
[hidden email]
http://lists.esug.org/mailman/listinfo/esug-list_lists.esug.org
Reply | Threaded
Open this post in threaded view
|

Re: ESUG SummerTalk - Fuel, binary object serializer

Mariano Martinez Peck


On Mon, May 30, 2011 at 8:27 AM, Paolo Bonzini <[hidden email]> wrote:
On 05/29/2011 08:51 PM, Mariano Martinez Peck wrote:
   But if I compare the serializing
   time and deserializing time of my thing, writing is about twice slower
   than reading.


Yes, I think that is the normal and more or less the expected difference
right now.


Do you want to make some comparisons with the GNU Smalltalk ObjectDumper?  I can help, or I can even make some comparisons myself if you give me some test Fuel code and an image to test with.


Hi Paolo. ObjectDumper was one of the serializers that we wanted to analyze also to get ideas and compare. This is why once I tried to install GNU Smalltalk in my  machine but I couldn't so I give up.
The problem to compare ObjectDumper is that we need to be able to run both in the same Dialect/VM. Otherwise the difference of the dialect or VM can change the results a lot.

Sorry for the offtopic but is there an easy way to install GNU Smalltalk in a Mac OS 10.6.7 ?

Thanks


--
Mariano
http://marianopeck.wordpress.com


_______________________________________________
Esug-list mailing list
[hidden email]
http://lists.esug.org/mailman/listinfo/esug-list_lists.esug.org
Reply | Threaded
Open this post in threaded view
|

Re: ESUG SummerTalk - Fuel, binary object serializer

Paolo Bonzini-2
On 05/30/2011 10:02 AM, Mariano Martinez Peck wrote:
> The problem to compare ObjectDumper is that we need to be able to run
> both in the same Dialect/VM. Otherwise the difference of the dialect
> or VM can change the results a lot.

That's true.

> Sorry for the offtopic but is there an easy way to install GNU
> Smalltalk in a Mac OS 10.6.7 ?

With 3.2.4 and MacPorts it should be easy.  You need to install the
libsigsegv port first.  It may not work right away if your compiler
doesn't look in MacPorts paths.  You can preempt the issue with an extra
option to the configure script:

  ./configure --with-system-libsigsegv=/opt/local/lib --prefix=/opt/local
  make
  sudo make install

(or /opt/local/lib64, I don't know :)).

ObjectDumper should be easy to port to Pharo.  Something like

gst-convert -f gst -F squeak /opt/local/smalltalk/kernel/ObjDumper.st |
   tr '\n' '\r' > objdumper.sq

should be a good start.  Check out category "private - binary I/O",
that's where changes might be made.  The resulting streams should be
portable between dialects, though I've never tried.

Testcases are old so they have not been converted to SUnit, but I'm sure
someone in the GNU Smalltalk community can help. :>

Lastly, it was written entirely by me, so I can relicense parts of it if
required.  Contact me off-list if you are interested.

HTH,

Paolo

_______________________________________________
Esug-list mailing list
[hidden email]
http://lists.esug.org/mailman/listinfo/esug-list_lists.esug.org
Reply | Threaded
Open this post in threaded view
|

Re: ESUG SummerTalk - Fuel, binary object serializer

stephane ducasse-2
In reply to this post by Mariano Martinez Peck
Yes if you do not use the same vm , the problem is that you will compare apple and orange.

Stef

>
>
> On Mon, May 30, 2011 at 8:27 AM, Paolo Bonzini <[hidden email]> wrote:
> On 05/29/2011 08:51 PM, Mariano Martinez Peck wrote:
>    But if I compare the serializing
>    time and deserializing time of my thing, writing is about twice slower
>    than reading.
>
>
> Yes, I think that is the normal and more or less the expected difference
> right now.
>
>
> Do you want to make some comparisons with the GNU Smalltalk ObjectDumper?  I can help, or I can even make some comparisons myself if you give me some test Fuel code and an image to test with.
>
>
> Hi Paolo. ObjectDumper was one of the serializers that we wanted to analyze also to get ideas and compare. This is why once I tried to install GNU Smalltalk in my  machine but I couldn't so I give up.
> The problem to compare ObjectDumper is that we need to be able to run both in the same Dialect/VM. Otherwise the difference of the dialect or VM can change the results a lot.
>
> Sorry for the offtopic but is there an easy way to install GNU Smalltalk in a Mac OS 10.6.7 ?
>
> Thanks
>
>
> --
> Mariano
> http://marianopeck.wordpress.com
>
> _______________________________________________
> Esug-list mailing list
> [hidden email]
> http://lists.esug.org/mailman/listinfo/esug-list_lists.esug.org


_______________________________________________
Esug-list mailing list
[hidden email]
http://lists.esug.org/mailman/listinfo/esug-list_lists.esug.org
Reply | Threaded
Open this post in threaded view
|

Re: ESUG SummerTalk - Fuel, binary object serializer

Mariano Martinez Peck
In reply to this post by Yoshiki Ohshima-2


On Sun, May 29, 2011 at 9:07 PM, Yoshiki Ohshima <[hidden email]> wrote:
At Sun, 29 May 2011 20:52:45 +0200,
Mariano Martinez Peck wrote:
>
> On Sun, May 29, 2011 at 8:45 PM, Yoshiki Ohshima <[hidden email]> wrote:
>
>     At Sat, 28 May 2011 09:33:27 +0200,
>     Stéphane Ducasse wrote:
>     >
>     > Yoshiki
>     >
>     > if you want to help testing, improving fuel you are welcome.
>     > The idea is to make it fast fast fast without vm support.
>
>      Yeah.  The reason for example I went to pad data to 4 bytes
>
> What do you mean by padding data to 4 bytes?  I don't understand :(

 Say a string has length of 5 ('abcde'), the data in file for it
would be "97 98 99 100 101 0 0 0".


Sorry but I don't understand then how that is related to "The reason for example I went to pad data to 4 bytes was that
there may be a clever trick I may be able to do to read data into
arrays "directly" and stuff, but did not get around implementing it."
 
>     was that
>     there may be a clever trick I may be able to do to read data into
>     arrays "directly" and stuff,
>
> what do you mean by that?  like ImageSegment does?  I mean, include also object headers in the stream and then avoid to recreate objects (using #basicNew) ?

 Not like ImageSegment, but more like #hackBits: reading from file
into a string object of right size.

I am not sure if I understand. For some kind of objects (those that are variable) we  store first its size and then at materialization time we directly create an object of that size using #basicNew:

but I guess you are talking about something else.

 
 For pointer arrays and actual
literal fields, I am not sure I can apply this however.

 I would not put object headers, as you would agree^^;

-- Yoshiki



--
Mariano
http://marianopeck.wordpress.com


_______________________________________________
Esug-list mailing list
[hidden email]
http://lists.esug.org/mailman/listinfo/esug-list_lists.esug.org
Reply | Threaded
Open this post in threaded view
|

Re: ESUG SummerTalk - Fuel, binary object serializer

Yoshiki Ohshima-2
At Tue, 31 May 2011 23:16:12 +0200,
Mariano Martinez Peck wrote:

>
> On Sun, May 29, 2011 at 9:07 PM, Yoshiki Ohshima <[hidden email]> wrote:
>
>     At Sun, 29 May 2011 20:52:45 +0200,
>     Mariano Martinez Peck wrote:
>     >
>     > On Sun, May 29, 2011 at 8:45 PM, Yoshiki Ohshima <[hidden email]> wrote:
>     >
>     >     At Sat, 28 May 2011 09:33:27 +0200,
>     >     Stéphane Ducasse wrote:
>     >     >
>     >     > Yoshiki
>     >     >
>     >     > if you want to help testing, improving fuel you are welcome.
>     >     > The idea is to make it fast fast fast without vm support.
>     >
>     >      Yeah.  The reason for example I went to pad data to 4 bytes
>     >
>     > What do you mean by padding data to 4 bytes?  I don't understand :(
>    
>      Say a string has length of 5 ('abcde'), the data in file for it
>     would be "97 98 99 100 101 0 0 0".
>
> Sorry but I don't understand then how that is related to "The reason for example I went to pad data to 4 bytes was that
> there may be a clever trick I may be able to do to read data into
> arrays "directly" and stuff, but did not get around implementing it."
>  
>
>     >     was that
>     >     there may be a clever trick I may be able to do to read data into
>     >     arrays "directly" and stuff,
>     >
>     > what do you mean by that?  like ImageSegment does?  I mean, include also object headers in the stream and then avoid to recreate objects (using #basicNew) ?
>    
>      Not like ImageSegment, but more like #hackBits: reading from file
>     into a string object of right size.
>
> I am not sure if I understand. For some kind of objects (those that are variable) we  store first its size and then at materialization time we directly create an object
> of that size using #basicNew:
>
> but I guess you are talking about something else.

  Hmm, I'm also confused.  You know that ByteStrings are
padded in the image, right?  And you see a user of #hackBits:,
#nextWordsInto:, right?  We can eliminate some buffer copying there.

  For example, when I run Fuel materializer, I see
PositionableStream>>nextString is used which does not do basicNew: of
(Byte)String but actually does extra copy (which can be easily
eliminated and would make it faster).

-- Yoshiki

_______________________________________________
Esug-list mailing list
[hidden email]
http://lists.esug.org/mailman/listinfo/esug-list_lists.esug.org
12