Smalltalk › Squeak › Squeak - Dev

How to store live objects in Monticello?

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

26 messages Options

Klaus D. Witzel

Re: CompileMethod limits

Hi Milan,

on Sun, 25 Feb 2007 00:03:22 +0100, you wrote:

> Hi Klaus,
> On 2007 February 24 04:07, Klaus D. Witzel wrote:
>> Hi Milan,
>>
>> on Sat, 24 Feb 2007 09:53:50 +0100, you wrote:
> <<snip>>
>> >
>> > There are limits in the CompileMethod code such as:
>> > nTemps > 63 ifTrue:
>> > [^ self error: 'Cannot compile -- too many temporary variables'].
>> > nLits > 255 ifTrue:
>> > [^ self error: 'Cannot compile -- too many literals variables'].
>> > A naive question, does this have to do with the way classes are
>> > represented in
>> > the VM?
>>
>> Since I am [in part] responsible for having the above checks in a
>> consistent way, here's my part of the answer:
>>
>> No. These limits exist because the header field in CompiledMethod has
>> room
>> for just these respective number of bits.
>>
>> Has nothing to do with, how classes are represented in the VM, but with,
>> how CompiledMethods are represented in the VM.
>
> Thanks for explaining it. Considering my zero level awareness of how
> image
> interacts with the VM, and that I never looked at VM code, I should take
> this
> to the newbies list .. but anyway. I assume this limit only applies to
> methods compiled dynamically using ClassDescription>>compile:classified:
> and
> friends, not to methods code that I type in browser, is that correct?

No :) There is but one CompiledMethod class so the way you compile doesn't
matter.

> Also, I
> wonder how other VMs handle this, is it a performance reason that Squeak
> (apparently) allocates fixed space to CompileMethod header (instead of
> maybe
> making it a linked list)?

You can see the four parts of CompiledMethod better than I can describe
them here, in a debugger inspect the thisContext item in the bottom right
pane (second from right); in that inspector inspect the CompiledMethod.
For example ('abc' at: 0) will start you the debugger, select the line
which says UndefinedObject>>DoIt.

The part you refer to is the literals array which is directly indexed by
bit quantities in the bytecode; there's no time for chasing a slow linked
list ;-)

/Klaus

> Milan
>>
>> > -# So I ended up "manually serializing" the ByteArray, esentially
>> as a
>> > literal String such as '80 75 3 4 20 0 0 0 0 0 76 134 203' in the
>> small
>> > example above. Then deserialized after the method answered.
>>
>> Correct. This way you have only one literal in the CompiledMethod,
>> instead
>> of one literals for each value less than -1 or greater than 2 :)
>>
>> /Klaus
>
>

Bert Freudenberg

Base64 vs. decimal (was: CompileMethod limits)

In reply to this post by Milan Zimmermann-2

On Feb 24, 2007, at 23:54 , Milan Zimmermann wrote:

> On 2007 February 24 06:08, Bert Freudenberg wrote:
>>
>> I was suggesting using #storeString on the Form, not the ByteArray.
>> This is more efficient because it uses words instead of bytes, and it
>> uses only one literal.
>
> I misunderstood ... by Form, do you mean the project contents read
> from file,
> I am not clear on that.

I think I misremembered ... you are trying to embed a project, right?
I thought it was a picture (class Form).

>> This way, the parsing work is done only once
>> when compiling. Reading decimal from a String at runtime is
>> particularly inefficient.
>> Don't do that ;)
>
> I live to learn :) - and thanks for help.
>
> Trying to be quick I misrepresented my wording about Decimals, the
> deserialization does:
>
> serializedByteArrayContentsAsString do: [ :b |
> (b = Character space)
> ifTrue: [coll addLast: token asInteger. token := ''.]
> ifFalse: [token := token,b asString.].]. "process last snipped"
> coll asByteArray.
> It is a hack and not very fast, but seems ok in the context of this
> being done
> once for all tests. Would this be considered slow, I am not sure
> what to
> compare it to. Maybe it is even slower than reading decimal from a
> String. I
> should really replace it.

Base64 has a space overhead of about 33%, it encodes 3 bytes in 4
characters. Printing each byte as decimal number plus a space adds
about 250% on average, each byte takes 2 to 4 characters to encode.
So space-wise base64 should be vastly more efficient. We can test that:

a := ((1 to: 100000) collect: [:i | i \\ 256]) asByteArray.
"encoding"
s1 := String streamContents: [:s | a do: [:each | s print: each;
space]].
s2 := (Base64MimeConverter mimeEncode: a readStream) contents.
"decoding"
t0 := [coll := OrderedCollection new. token := ''.
s1 do: [ :b | (b = Character space)
ifTrue: [coll addLast: token asInteger. token := ''.]
ifFalse: [token := token,b asString.].].
a0:= coll asByteArray] timeToRun.
t1 := [r1 := s1 readStream.
a1 := ByteArray streamContents: [:s |
[r1 atEnd] whileFalse: [s nextPut: (Integer readFrom: r1). r1 skip:
1]]] timeToRun.
t2 := [a2 := (Base64MimeConverter mimeDecodeToBytes: s2 readStream)
contents] timeToRun.
{'Original'. a0=a. s1 size. t0. 'Decimal'. a1=a. s1 size. t1.
'Base64'. a2=a. s2 size. t2}

#('Original' true 356992 2334 'Decimal' true 356992 1064 'Base64'
true 135187 103)

So your code (t0) can be optimized a bit (t1), making it half as
slow. But doing base64 (t2) is an additional 10 times faster.

So ... 10 times faster ... using a third of the space ... base64 wins
hands down, IMHO ;)

- Bert -

Milan Zimmermann-2

Re: Base64 vs. decimal (was: CompileMethod limits)

On 2007 February 25 08:09, Bert Freudenberg wrote:

> On Feb 24, 2007, at 23:54 , Milan Zimmermann wrote:
> > On 2007 February 24 06:08, Bert Freudenberg wrote:
> >> I was suggesting using #storeString on the Form, not the ByteArray.
> >> This is more efficient because it uses words instead of bytes, and it
> >> uses only one literal.
> >
> > I misunderstood ... by Form, do you mean the project contents read
> > from file,
> > I am not clear on that.
>
> I think I misremembered ... you are trying to embed a project, right?
> I thought it was a picture (class Form).

Yes, a project the is read from a pr file.

<<snip>>

> Base64 has a space overhead of about 33%, it encodes 3 bytes in 4
> characters. Printing each byte as decimal number plus a space adds
> about 250% on average, each byte takes 2 to 4 characters to encode.
> So space-wise base64 should be vastly more efficient. We can test that:
>
> a := ((1 to: 100000) collect: [:i | i \\ 256]) asByteArray.
> "encoding"
> s1 := String streamContents: [:s | a do: [:each | s print: each;
> space]].
> s2 := (Base64MimeConverter mimeEncode: a readStream) contents.
> "decoding"
> t0 := [coll := OrderedCollection new. token := ''.
> s1 do: [ :b | (b = Character space)
> ifTrue: [coll addLast: token asInteger. token := ''.]
> ifFalse: [token := token,b asString.].].
> a0:= coll asByteArray] timeToRun.
> t1 := [r1 := s1 readStream.
> a1 := ByteArray streamContents: [:s |
> [r1 atEnd] whileFalse: [s nextPut: (Integer readFrom: r1). r1 skip:
> 1]]] timeToRun.
> t2 := [a2 := (Base64MimeConverter mimeDecodeToBytes: s2 readStream)
> contents] timeToRun.
> {'Original'. a0=a. s1 size. t0. 'Decimal'. a1=a. s1 size. t1.
> 'Base64'. a2=a. s2 size. t2}
>
> #('Original' true 356992 2334 'Decimal' true 356992 1064 'Base64'
> true 135187 103)

I will not argue with 10x performance gain and provided code on top of it :)
and will replace my code with Base64MimeConverter

Thanks!

Milan

>
> So your code (t0) can be optimized a bit (t1), making it half as
> slow. But doing base64 (t2) is an additional 10 times faster.
>
> So ... 10 times faster ... using a third of the space ... base64 wins
> hands down, IMHO ;)
>
> - Bert -

Milan Zimmermann-2

Re: CompileMethod limits

In reply to this post by Klaus D. Witzel

Hi Klaus,

On 2007 February 24 20:14, Klaus D. Witzel wrote:
<<>>

>
> No :) There is but one CompiledMethod class so the way you compile doesn't
> matter.
>

ok thanks that makes sense, i will also play with just pasting the same method
content into browser

> > Also, I
> > wonder how other VMs handle this, is it a performance reason that Squeak
> > (apparently) allocates fixed space to CompileMethod header (instead of
> > maybe
> > making it a linked list)?
>
> You can see the four parts of CompiledMethod better than I can describe
> them here, in a debugger inspect the thisContext item in the bottom right
> pane (second from right); in that inspector inspect the CompiledMethod.
> For example ('abc' at: 0) will start you the debugger, select the line
> which says UndefinedObject>>DoIt.

ah that is interesting, I never drilled in thisContext in debugger (or looked
at the bytecodes).

>
> The part you refer to is the literals array which is directly indexed by
> bit quantities in the bytecode; there's no time for chasing a slow linked
> list ;-)

Thanks for the tips, (much Squeak to learn still),

Milan

>
> /Klaus
>
> > Milan
> >
> >> > -# So I ended up "manually serializing" the ByteArray, esentially
> >>
> >> as a
> >>
> >> > literal String such as '80 75 3 4 20 0 0 0 0 0 76 134 203' in the
> >>
> >> small
> >>
> >> > example above. Then deserialized after the method answered.
> >>
> >> Correct. This way you have only one literal in the CompiledMethod,
> >> instead
> >> of one literals for each value less than -1 or greater than 2 :)
> >>
> >> /Klaus

timrowledge

Re: CompileMethod limits

You might well find

http://www.rowledge.org/tim/squeak/OE-Tour.html

useful as well.

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Computer Science: solving today's problems tomorrow.

Milan Zimmermann-2

Re: CompileMethod limits

On 2007 February 27 00:57, tim Rowledge wrote:
> You might well find
>
> http://www.rowledge.org/tim/squeak/OE-Tour.html

thanks Tim.

>
> useful as well.
>
> tim
> --
> tim Rowledge; [hidden email]; http://www.rowledge.org/tim
> Computer Science: solving today's problems tomorrow.