Smalltalk › Squeak › Squeak VM

Compact class question

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

12 messages Options

Gabriel Hernán Barbuto

Compact class question

Hi

I am working on writing an image to disk. If I understand correctly,
the size in words field in the one word header must take into account
the word for the header. I have a CompiledMethod instance that has 63
words of content plus at least one header word for a total of 64
words.

I think that I should write a three word header for this instance even
though its class is compact. Is this correct? Is it fine to have an
instance of compact class that doesn't have a one word header?

Thanks in advance.
Gabriel

Bert Freudenberg

Re: Compact class question

On 06.12.2010, at 14:19, Gabriel Hernán Barbuto wrote:

>
> Hi
>
> I am working on writing an image to disk. If I understand correctly,
> the size in words field in the one word header must take into account
> the word for the header. I have a CompiledMethod instance that has 63
> words of content plus at least one header word for a total of 64
> words.
>
> I think that I should write a three word header for this instance even
> though its class is compact. Is this correct? Is it fine to have an
> instance of compact class that doesn't have a one word header?

I think it must have a one-word header.

Are you using MicroSqueak for writing the image? It works fine, including writing compact classes. But be aware John's code had a bug when writing methods that were exactly 63 bytes long. I sent him a fix in the mean time.

- Bert -

Gabriel Hernán Barbuto

Re: Compact class question

Hi Bert

Thanks for your response. I am using John's code and I think I am
seeing the same bug. But I think the problem is when the object is 63
words long.

Ben found a method that produced a fail in an assertion. It's a
CompiledMethod instance that occupies 63 words. When the code makes
the calculation to see how many words it needs. It calculates that it
needs 1 word for the header and a total of 64 words for the object in
total. Because it needs to also take into account the word for the
header.

I don't understand how you can encoded this object with a one word
header. I don't know about your fix. But I think the problem is that
this object cannot have a one word header. Because the size field only
allows for values up to 63 words and this value has to take the base
header word into account.

The problem is that John's code calculate the total words that an
object requires in two different places, and does it slightly
different in each one.

In MicroSqueakImageBuilder>>#headerAndTotalWords he calculates the
size of the object like this(CompiledMethod is variable and is bytes):

contents = instSize + (basicSize + 3 // 4) + extraWords

basicSize + 3 // 4 is called indexableWords in the other method.

I think the problem is in the following expression:

headerWords :=
contentsWords > 63
ifTrue: [3]
ifFalse: [(cl indexIfCompact > 0) ifTrue: [1] ifFalse: [2]].

It should check against 62, because if the object has 63 words of
content, it cannot have a one word header.

In MicroSqueakImageBuilder>>#writeObjHeaderFor:map:on:

totalWords := instSize + indexableWords + extraWords + 1.

The extra +1 in the last part of the expression make this particular
object to have a totalWords of 64, and so I think it cannot have a one
word header.

totalWords > 63
ifTrue: [ "3-word header"
ifFalse: [
cl indexIfCompact = 0
ifTrue: [ "2-word header"
ifFalse: [ "1-word header"]]

I think the object should have a 3 word header. Can you explain me how
to encode the size of this object in a one word header. Because I
don't understand how to do this. Thanks in advance.

Bests
Gabriel

On Mon, Dec 6, 2010 at 3:31 PM, Bert Freudenberg <[hidden email]> wrote:

>
>
> On 06.12.2010, at 14:19, Gabriel Hernán Barbuto wrote:
>
>>
>> Hi
>>
>> I am working on writing an image to disk. If I understand correctly,
>> the size in words field in the one word header must take into account
>> the word for the header. I have a CompiledMethod instance that has 63
>> words of content plus at least one header word for a total of 64
>> words.
>>
>> I think that I should write a three word header for this instance even
>> though its class is compact. Is this correct? Is it fine to have an
>> instance of compact class that doesn't have a one word header?
>
> I think it must have a one-word header.
>
> Are you using MicroSqueak for writing the image? It works fine, including writing compact classes. But be aware John's code had a bug when writing methods that were exactly 63 bytes long. I sent him a fix in the mean time.
>
> - Bert -
>
>

Yoshiki Ohshima-2

Re: Compact class question

At Mon, 6 Dec 2010 16:10:22 +0100,
Gabriel Hernán Barbuto wrote:

>
>
> Hi Bert
>
> Thanks for your response. I am using John's code and I think I am
> seeing the same bug. But I think the problem is when the object is 63
> words long.
>
> Ben found a method that produced a fail in an assertion. It's a
> CompiledMethod instance that occupies 63 words. When the code makes
> the calculation to see how many words it needs. It calculates that it
> needs 1 word for the header and a total of 64 words for the object in
> total. Because it needs to also take into account the word for the
> header.
>
> I don't understand how you can encoded this object with a one word
> header. I don't know about your fix. But I think the problem is that
> this object cannot have a one word header. Because the size field only
> allows for values up to 63 words and this value has to take the base
> header word into account.
>
> The problem is that John's code calculate the total words that an
> object requires in two different places, and does it slightly
> different in each one.
>
> In MicroSqueakImageBuilder>>#headerAndTotalWords he calculates the
> size of the object like this(CompiledMethod is variable and is bytes):
>
> contents = instSize + (basicSize + 3 // 4) + extraWords
>
> basicSize + 3 // 4 is called indexableWords in the other method.
>
> I think the problem is in the following expression:
>
> headerWords :=
> contentsWords > 63
> ifTrue: [3]
> ifFalse: [(cl indexIfCompact > 0) ifTrue: [1] ifFalse: [2]].
>
> It should check against 62, because if the object has 63 words of
> content, it cannot have a one word header.

That would make it a two-off bug^^; It should check against 63,
with >=.

> In MicroSqueakImageBuilder>>#writeObjHeaderFor:map:on:
>
> totalWords := instSize + indexableWords + extraWords + 1.
>
> The extra +1 in the last part of the expression make this particular
> object to have a totalWords of 64, and so I think it cannot have a one
> word header.
>
> totalWords > 63
> ifTrue: [ "3-word header"
> ifFalse: [
> cl indexIfCompact = 0
> ifTrue: [ "2-word header"
> ifFalse: [ "1-word header"]]
>
> I think the object should have a 3 word header. Can you explain me how
> to encode the size of this object in a one word header. Because I
> don't understand how to do this. Thanks in advance.

63 fits within the basic header's size field.

(We're having some fun with MicroSqueak. It would be very
interesting to hear what you guys are up to..)

-- Yoshiki

Mariano Martinez Peck

Re: Compact class question

Gabriel, OT, in Pharo 1.2 I added the method #sizeInMemory. You may want to use that directly ;)

Cheers

Mariano

On Mon, Dec 6, 2010 at 5:01 PM, Yoshiki Ohshima <[hidden email]> wrote:

At Mon, 6 Dec 2010 16:10:22 +0100,

Gabriel Hernán Barbuto wrote:
>
>
> Hi Bert
>
> Thanks for your response. I am using John's code and I think I am
> seeing the same bug. But I think the problem is when the object is 63
> words long.
>
> Ben found a method that produced a fail in an assertion. It's a
> CompiledMethod instance that occupies 63 words. When the code makes
> the calculation to see how many words it needs. It calculates that it
> needs 1 word for the header and a total of 64 words for the object in
> total. Because it needs to also take into account the word for the
> header.
>
> I don't understand how you can encoded this object with a one word
> header. I don't know about your fix. But I think the problem is that
> this object cannot have a one word header. Because the size field only
> allows for values up to 63 words and this value has to take the base
> header word into account.
>
> The problem is that John's code calculate the total words that an
> object requires in two different places, and does it slightly
> different in each one.
>
> In MicroSqueakImageBuilder>>#headerAndTotalWords he calculates the
> size of the object like this(CompiledMethod is variable and is bytes):
>
> contents = instSize + (basicSize + 3 // 4) + extraWords
>
> basicSize + 3 // 4 is called indexableWords in the other method.
>
> I think the problem is in the following expression:
>
> headerWords :=
> contentsWords > 63
> ifTrue: [3]
> ifFalse: [(cl indexIfCompact > 0) ifTrue: [1] ifFalse: [2]].
>
> It should check against 62, because if the object has 63 words of
> content, it cannot have a one word header.

That would make it a two-off bug^^; It should check against 63,
with >=.

> In MicroSqueakImageBuilder>>#writeObjHeaderFor:map:on:
>
> totalWords := instSize + indexableWords + extraWords + 1.
>
> The extra +1 in the last part of the expression make this particular
> object to have a totalWords of 64, and so I think it cannot have a one
> word header.
>
> totalWords > 63
> ifTrue: [ "3-word header"
> ifFalse: [
> cl indexIfCompact = 0
> ifTrue: [ "2-word header"
> ifFalse: [ "1-word header"]]
>
> I think the object should have a 3 word header. Can you explain me how
> to encode the size of this object in a one word header. Because I
> don't understand how to do this. Thanks in advance.

63 fits within the basic header's size field.

(We're having some fun with MicroSqueak. It would be very
interesting to hear what you guys are up to..)

-- Yoshiki

Gabriel Hernán Barbuto

Re: Compact class question

In reply to this post by Yoshiki Ohshima-2

Hi Yoshiki

Maybe I have not made myself clear about which expression my comment
referred to. Read my mail again, but I will copy it here again:

headerWords :=
contentsWords > 63
ifTrue: [3]
ifFalse: [(cl indexIfCompact > 0) ifTrue: [1] ifFalse: [2]].

Replacing the 63 for a 62 will make the trick and besides if
contentsWords is greater than 62 it should be greater than or equal to
63. So what you propose is the same. But be careful that it's not in
totalWords expression.

I think this object should have a 3-word header. What do you think about this?

With respect to what we are doing. We are porting MicroSqueak to
Pharo. MicroSqueak could be useful for teaching. The next step(we are
also working in parallel on this) is to build kernel images but
different from MicroSqueak. In order to do that, we need to remove all
the bad dependencies on the kernel classes and clean up the Pharo
image. What are you doing with MicroSqueak?

Gabriel

On Mon, Dec 6, 2010 at 5:01 PM, Yoshiki Ohshima <[hidden email]> wrote:

>
> At Mon, 6 Dec 2010 16:10:22 +0100,
> Gabriel Hernán Barbuto wrote:
>>
>>
>> Hi Bert
>>
>> Thanks for your response. I am using John's code and I think I am
>> seeing the same bug. But I think the problem is when the object is 63
>> words long.
>>
>> Ben found a method that produced a fail in an assertion. It's a
>> CompiledMethod instance that occupies 63 words. When the code makes
>> the calculation to see how many words it needs. It calculates that it
>> needs 1 word for the header and a total of 64 words for the object in
>> total. Because it needs to also take into account the word for the
>> header.
>>
>> I don't understand how you can encoded this object with a one word
>> header. I don't know about your fix. But I think the problem is that
>> this object cannot have a one word header. Because the size field only
>> allows for values up to 63 words and this value has to take the base
>> header word into account.
>>
>> The problem is that John's code calculate the total words that an
>> object requires in two different places, and does it slightly
>> different in each one.
>>
>> In MicroSqueakImageBuilder>>#headerAndTotalWords he calculates the
>> size of the object like this(CompiledMethod is variable and is bytes):
>>
>> contents = instSize + (basicSize + 3 // 4) + extraWords
>>
>> basicSize + 3 // 4 is called indexableWords in the other method.
>>
>> I think the problem is in the following expression:
>>
>> headerWords :=
>> contentsWords > 63
>> ifTrue: [3]
>> ifFalse: [(cl indexIfCompact > 0) ifTrue: [1] ifFalse: [2]].
>>
>> It should check against 62, because if the object has 63 words of
>> content, it cannot have a one word header.
>
> That would make it a two-off bug^^; It should check against 63,
> with >=.
>
>> In MicroSqueakImageBuilder>>#writeObjHeaderFor:map:on:
>>
>> totalWords := instSize + indexableWords + extraWords + 1.
>>
>> The extra +1 in the last part of the expression make this particular
>> object to have a totalWords of 64, and so I think it cannot have a one
>> word header.
>>
>> totalWords > 63
>> ifTrue: [ "3-word header"
>> ifFalse: [
>> cl indexIfCompact = 0
>> ifTrue: [ "2-word header"
>> ifFalse: [ "1-word header"]]
>>
>> I think the object should have a 3 word header. Can you explain me how
>> to encode the size of this object in a one word header. Because I
>> don't understand how to do this. Thanks in advance.
>
> 63 fits within the basic header's size field.
>
> (We're having some fun with MicroSqueak. It would be very
> interesting to hear what you guys are up to..)
>
> -- Yoshiki
>

Yoshiki Ohshima-2

Re: Compact class question

At Tue, 7 Dec 2010 10:53:12 +0100,
Gabriel Hernán Barbuto wrote:

>
>
> Hi Yoshiki
>
> Maybe I have not made myself clear about which expression my comment
> referred to. Read my mail again, but I will copy it here again:
>
> headerWords :=
> contentsWords > 63
> ifTrue: [3]
> ifFalse: [(cl indexIfCompact > 0) ifTrue: [1] ifFalse: [2]].
>
> Replacing the 63 for a 62 will make the trick and besides if
> contentsWords is greater than 62 it should be greater than or equal to
> 63. So what you propose is the same. But be careful that it's not in
> totalWords expression.
>
> I think this object should have a 3-word header. What do you think about this?

Yes, I think so.

I didn't quite understand the intent of original question ("Is it
fine to have an instance of compact class that doesn't have a one word
header?"). The answer, as Bert wrote, an object needs to have the
base header, if the "one word header" refers to the base header.

> With respect to what we are doing. We are porting MicroSqueak to
> Pharo. MicroSqueak could be useful for teaching. The next step(we are
> also working in parallel on this) is to build kernel images but
> different from MicroSqueak. In order to do that, we need to remove all
> the bad dependencies on the kernel classes and clean up the Pharo
> image.
>
> What are you doing with MicroSqueak?

Cool, and thank you. In some ways, what we are doing is similar; we
wrote an app in Squeak but trying to understand what other parts of
Squeak the app is using and going from bottom up sounds like an
intersting idea.

-- Yoshiki

Gabriel Hernán Barbuto

Re: Compact class question

On Tue, Dec 7, 2010 at 11:30 AM, Yoshiki Ohshima <[hidden email]> wrote:

>
> At Tue, 7 Dec 2010 10:53:12 +0100,
> Gabriel Hernán Barbuto wrote:
>>
>>
>> Hi Yoshiki
>>
>> Maybe I have not made myself clear about which expression my comment
>> referred to. Read my mail again, but I will copy it here again:
>>
>> headerWords :=
>> contentsWords > 63
>> ifTrue: [3]
>> ifFalse: [(cl indexIfCompact > 0) ifTrue: [1] ifFalse: [2]].
>>
>> Replacing the 63 for a 62 will make the trick and besides if
>> contentsWords is greater than 62 it should be greater than or equal to
>> 63. So what you propose is the same. But be careful that it's not in
>> totalWords expression.
>>
>> I think this object should have a 3-word header. What do you think about this?
>
> Yes, I think so.
>
> I didn't quite understand the intent of original question ("Is it
> fine to have an instance of compact class that doesn't have a one word
> header?"). The answer, as Bert wrote, an object needs to have the
> base header, if the "one word header" refers to the base header.
>

My question was if it is possible to have an instance of a compact
class with a 3-word header. I thought that all compact classes
instances should have only a 1-word header. I understand that the base
header will always be there. But I was not sure if it was legal to
have more than the base header for compact classes instances.

Hope this makes my question clearer.

Gabriel

>> With respect to what we are doing. We are porting MicroSqueak to
>> Pharo. MicroSqueak could be useful for teaching. The next step(we are
>> also working in parallel on this) is to build kernel images but
>> different from MicroSqueak. In order to do that, we need to remove all
>> the bad dependencies on the kernel classes and clean up the Pharo
>> image.
>>
>> What are you doing with MicroSqueak?
>
> Cool, and thank you. In some ways, what we are doing is similar; we
> wrote an app in Squeak but trying to understand what other parts of
> Squeak the app is using and going from bottom up sounds like an
> intersting idea.
>
> -- Yoshiki
>

Igor Stasenko

Re: Compact class question

On 7 December 2010 12:43, Gabriel Hernán Barbuto <[hidden email]> wrote:

>
> On Tue, Dec 7, 2010 at 11:30 AM, Yoshiki Ohshima <[hidden email]> wrote:
>>
>> At Tue, 7 Dec 2010 10:53:12 +0100,
>> Gabriel Hernán Barbuto wrote:
>>>
>>>
>>> Hi Yoshiki
>>>
>>> Maybe I have not made myself clear about which expression my comment
>>> referred to. Read my mail again, but I will copy it here again:
>>>
>>> headerWords :=
>>> contentsWords > 63
>>> ifTrue: [3]
>>> ifFalse: [(cl indexIfCompact > 0) ifTrue: [1] ifFalse: [2]].
>>>
>>> Replacing the 63 for a 62 will make the trick and besides if
>>> contentsWords is greater than 62 it should be greater than or equal to
>>> 63. So what you propose is the same. But be careful that it's not in
>>> totalWords expression.
>>>
>>> I think this object should have a 3-word header. What do you think about this?
>>
>> Yes, I think so.
>>
>> I didn't quite understand the intent of original question ("Is it
>> fine to have an instance of compact class that doesn't have a one word
>> header?"). The answer, as Bert wrote, an object needs to have the
>> base header, if the "one word header" refers to the base header.
>>
>
> My question was if it is possible to have an instance of a compact
> class with a 3-word header. I thought that all compact classes
> instances should have only a 1-word header. I understand that the base
> header will always be there. But I was not sure if it was legal to
> have more than the base header for compact classes instances.
>
> Hope this makes my question clearer.

Array is a compact class, but it is no doubt allows you to have more than
63 words (62 elements in array).
You can check it easily by issuing:

Array new: 10000

:)

>
> Gabriel
>
>>> With respect to what we are doing. We are porting MicroSqueak to
>>> Pharo. MicroSqueak could be useful for teaching. The next step(we are
>>> also working in parallel on this) is to build kernel images but
>>> different from MicroSqueak. In order to do that, we need to remove all
>>> the bad dependencies on the kernel classes and clean up the Pharo
>>> image.
>>>
>>> What are you doing with MicroSqueak?
>>
>> Cool, and thank you. In some ways, what we are doing is similar; we
>> wrote an app in Squeak but trying to understand what other parts of
>> Squeak the app is using and going from bottom up sounds like an
>> intersting idea.
>>
>> -- Yoshiki
>>
>

--
Best regards,
Igor Stasenko AKA sig.

Bert Freudenberg

Re: Compact class question

In reply to this post by Gabriel Hernán Barbuto

Find my fix attached. And you are right, it was at 63 words not bytes.

The first pass used 3-word headers only if contentsSize > 63 words, but the base header's size (1 word) needs to be included in that calculation.

Here's my isolated test method:

foo
"image build fails if size is 253 to 256 bytes"
"remove return to change size by 1, add/remove zeroes to change by 2"
^0+0+0+0+1+2+3+4+5+6+7+8+9+10+11+12+13+14+15+16+17+18+19+20
+21+22+23+24+25+26+27+28+29+30+31+32+33+34+35+36+37+38+39

- Bert -

On 06.12.2010, at 07:10, Gabriel Hernán Barbuto wrote:

>
> Hi Bert
>
> Thanks for your response. I am using John's code and I think I am
> seeing the same bug. But I think the problem is when the object is 63
> words long.
>
> Ben found a method that produced a fail in an assertion. It's a
> CompiledMethod instance that occupies 63 words. When the code makes
> the calculation to see how many words it needs. It calculates that it
> needs 1 word for the header and a total of 64 words for the object in
> total. Because it needs to also take into account the word for the
> header.
>
> I don't understand how you can encoded this object with a one word
> header. I don't know about your fix. But I think the problem is that
> this object cannot have a one word header. Because the size field only
> allows for values up to 63 words and this value has to take the base
> header word into account.
>
> The problem is that John's code calculate the total words that an
> object requires in two different places, and does it slightly
> different in each one.
>
> In MicroSqueakImageBuilder>>#headerAndTotalWords he calculates the
> size of the object like this(CompiledMethod is variable and is bytes):
>
> contents = instSize + (basicSize + 3 // 4) + extraWords
>
> basicSize + 3 // 4 is called indexableWords in the other method.
>
> I think the problem is in the following expression:
>
> headerWords :=
> contentsWords > 63
> ifTrue: [3]
> ifFalse: [(cl indexIfCompact > 0) ifTrue: [1] ifFalse: [2]].
>
> It should check against 62, because if the object has 63 words of
> content, it cannot have a one word header.
>
>
> In MicroSqueakImageBuilder>>#writeObjHeaderFor:map:on:
>
> totalWords := instSize + indexableWords + extraWords + 1.
>
> The extra +1 in the last part of the expression make this particular
> object to have a totalWords of 64, and so I think it cannot have a one
> word header.
>
> totalWords > 63
> ifTrue: [ "3-word header"
> ifFalse: [
> cl indexIfCompact = 0
> ifTrue: [ "2-word header"
> ifFalse: [ "1-word header"]]
>
> I think the object should have a 3 word header. Can you explain me how
> to encode the size of this object in a one word header. Because I
> don't understand how to do this. Thanks in advance.
>
> Bests
> Gabriel
>
>
> On Mon, Dec 6, 2010 at 3:31 PM, Bert Freudenberg <[hidden email]> wrote:
>>
>>
>> On 06.12.2010, at 14:19, Gabriel Hernán Barbuto wrote:
>>
>>>
>>> Hi
>>>
>>> I am working on writing an image to disk. If I understand correctly,
>>> the size in words field in the one word header must take into account
>>> the word for the header. I have a CompiledMethod instance that has 63
>>> words of content plus at least one header word for a total of 64
>>> words.
>>>
>>> I think that I should write a three word header for this instance even
>>> though its class is compact. Is this correct? Is it fine to have an
>>> instance of compact class that doesn't have a one word header?
>>
>> I think it must have a one-word header.
>>
>> Are you using MicroSqueak for writing the image? It works fine, including writing compact classes. But be aware John's code had a bug when writing methods that were exactly 63 bytes long. I sent him a fix in the mean time.
>>
>> - Bert -
>>
>>

MicroSqueakImageBuilder-headerAndTotalWordsFor.st (1K) Download Attachment

stephane ducasse-2

Re: Compact class question

In reply to this post by Yoshiki Ohshima-2

Hi Yoshiki

Our goal is to bootstrap the system so in the process we are cleaning a lot of dependencies
and misplaced behavior. I did not get what was your approach.
We tried to several onces before microsqueak came in our radar.
- selecting classes and recompiling them in another environment filled with stub.
- building analysis tools on top of Moose to identify dependencies

Stef

Yoshiki Ohshima-2

Re: Compact class question

At Wed, 8 Dec 2010 18:46:20 +0100,
stephane ducasse wrote:

>
> Hi Yoshiki
>
> >> With respect to what we are doing. We are porting MicroSqueak to
> >> Pharo. MicroSqueak could be useful for teaching. The next step(we are
> >> also working in parallel on this) is to build kernel images but
> >> different from MicroSqueak. In order to do that, we need to remove all
> >> the bad dependencies on the kernel classes and clean up the Pharo
> >> image.
> >>
> >> What are you doing with MicroSqueak?
> >
> > Cool, and thank you. In some ways, what we are doing is similar; we
> > wrote an app in Squeak but trying to understand what other parts of
> > Squeak the app is using and going from bottom up sounds like an
> > intersting idea.
>
> Our goal is to bootstrap the system so in the process we are cleaning a lot of dependencies
> and misplaced behavior. I did not get what was your approach.
> We tried to several onces before microsqueak came in our radar.
> - selecting classes and recompiling them in another environment filled with stub.
> - building analysis tools on top of Moose to identify dependencies

What "we" do is to manually build an app on top of an MicroSqueak
image so that we can say everything in the image is needed for the app
(give or take). Not very general, but useful exercise for us
nontheless. On the course, we made Compiler work without any of these
stuff.

Similar to your second idea but I am more interested in whether we
can bootstrap Squeak image from text-based programming language like
C, I am writing a Squeak bytecode compiler in a non-Squeak language
(as a recreational programming), which generates a binary file with
bytecode and spec of unbound literals. (The prototype of this
compiler is in OMeta2/Squeak so it is still in Squeak, but should be
easily portable as it does not use any "Squeak-like" feature). The
resulting binary file can be loaded to MicroSqueak.

-- Yoshiki