About the image

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
18 messages Options
Reply | Threaded
Open this post in threaded view
|

About the image

Nicolás Paez
 
Hi, I have just began to work at this low level and I have 2 questions:

- What is the minimal object structure I should implement from VM perspective?

- Where are defined the 3 formats of objects representation?

Thanks,
Reply | Threaded
Open this post in threaded view
|

Re: About the image

Bert Freudenberg


On 17.08.2010, at 09:11, Nicolás Paez wrote:

> Hi, I have just began to work at this low level and I have 2 questions:

Nice. But you need to give us a little more context. What are you trying to do?

> - What is the minimal object structure I should implement from VM perspective?

Normally you do not have to implement this, as the interpreter is shared between all platforms.

> - Where are defined the 3 formats of objects representation?

Not sure what you mean. ObjectMemory>>formatOf: defines the class format bits in the header.

Or do you mean the 3 header formats? Read the class comment of ObjectMemory.

Also, there is a nice explanation in section A.1 of Tim's chapter in the NuBlue book:

http://www.rowledge.org/tim/squeak/assets/OE-Tour.pdf

- Bert -

Reply | Threaded
Open this post in threaded view
|

Re: About the image

stephane ducasse-2



>> Hi, I have just began to work at this low level and I have 2 questions:
>
> Nice. But you need to give us a little more context. What are you trying to do?

Bootstrap
and shaking/documenting the hidden assumptions on the way.
>
>> - What is the minimal object structure I should implement from VM perspective?
>
> Normally you do not have to implement this, as the interpreter is shared between all platforms.

yes we know. Nicolas read the chapter of tim but we wanted to check if this is still up to date.
My original question was more: where can I find the class object VM interpretation (that the first slot should be superclass, format....)
because I remember that we could not add an instance variable into behavior when we did traits.


>> - Where are defined the 3 formats of objects representation?
>
> Not sure what you mean. ObjectMemory>>formatOf: defines the class format bits in the header.

Yes we saw that.

> Or do you mean the 3 header formats? Read the class comment of ObjectMemory.
>
> Also, there is a nice explanation in section A.1 of Tim's chapter in the NuBlue book:
>
> http://www.rowledge.org/tim/squeak/assets/OE-Tour.pdf

Another question was where can we get a description of the image format.
I was planning to read the VM C code included the generated one.

Stef
Reply | Threaded
Open this post in threaded view
|

Re: About the image

Bert Freudenberg

On 17.08.2010, at 13:27, stephane ducasse wrote:

>>> Hi, I have just began to work at this low level and I have 2 questions:
>>
>> Nice. But you need to give us a little more context. What are you trying to do?
>
> Bootstrap

Building an image from first principles? Nice :)

> and shaking/documenting the hidden assumptions on the way.

Excellent!

>>> - What is the minimal object structure I should implement from VM perspective?
>>
>> Normally you do not have to implement this, as the interpreter is shared between all platforms.
>
> yes we know. Nicolas read the chapter of tim but we wanted to check if this is still up to date.
> My original question was more: where can I find the class object VM interpretation (that the first slot should be superclass, format....)
> because I remember that we could not add an instance variable into behavior when we did traits.

Ah. That's a more concrete question :)

I thought the three inst vars in Behavior was all the VM knew and cared about? I'd consider everything else a bug ;)

>>> - Where are defined the 3 formats of objects representation?
>>
>> Not sure what you mean. ObjectMemory>>formatOf: defines the class format bits in the header.
>
> Yes we saw that.
>
>> Or do you mean the 3 header formats? Read the class comment of ObjectMemory.
>>
>> Also, there is a nice explanation in section A.1 of Tim's chapter in the NuBlue book:
>>
>> http://www.rowledge.org/tim/squeak/assets/OE-Tour.pdf
>
> Another question was where can we get a description of the image format.
> I was planning to read the VM C code included the generated one.

The image is just a header plus a memory dump of the object memory. The header is written in Interpreter>>writeImageFileIO: and read in Interpreter>>readImageFromFile:HeapSize:StartingAt:.

- Bert -


Reply | Threaded
Open this post in threaded view
|

Re: About the image

stephane ducasse-2

>>
>> Another question was where can we get a description of the image format.
>> I was planning to read the VM C code included the generated one.
>
> The image is just a header plus a memory dump of the object memory. The header is written in Interpreter>>writeImageFileIO: and read in Interpreter>>readImageFromFile:HeapSize:StartingAt:.

Yes I read the last one in Slang
Probably I should read it in C because Slang is sometimes less friendly than C.


>
> - Bert -
>
>

Reply | Threaded
Open this post in threaded view
|

Re: About the image

Eliot Miranda-2
In reply to this post by Bert Freudenberg
 


On Tue, Aug 17, 2010 at 5:38 AM, Bert Freudenberg <[hidden email]> wrote:

On 17.08.2010, at 13:27, stephane ducasse wrote:

>>> Hi, I have just began to work at this low level and I have 2 questions:
>>
>> Nice. But you need to give us a little more context. What are you trying to do?
>
> Bootstrap

Building an image from first principles? Nice :)

> and shaking/documenting the hidden assumptions on the way.

Excellent!

>>> - What is the minimal object structure I should implement from VM perspective?
>>
>> Normally you do not have to implement this, as the interpreter is shared between all platforms.
>
> yes we know. Nicolas read the chapter of tim but we wanted to check if this is still up to date.
> My original question was more: where can I find the class object VM interpretation (that the first slot should be superclass, format....)
> because I remember that we could not add an instance variable into behavior when we did traits.

Ah. That's a more concrete question :)

I thought the three inst vars in Behavior was all the VM knew and cared about? I'd consider everything else a bug ;)

That's right.  See all implementors of initializeClassIndices:

BTW, you /can/ add an inst var to Behavior.  Try and evaluate

Object subclass: #Behavior
instanceVariableNames: 'superclass methodDict format extra'
classVariableNames: 'ObsoleteSubclasses'
poolDictionaries: ''
category: 'Kernel-Classes'

Works for me in a 4.1 derived image.
 

BTW Stéphane,  IMO the VM's Slang code is far friendlier than the generated C.  The Slang translator is the thing that's unfriendly.  It doesn't do much verification and so when it mistranslates it can be hard to figure out what's going on.  but the Slang/Smalltalk that implements the VM is fine. 

>>> - Where are defined the 3 formats of objects representation?
>>
>> Not sure what you mean. ObjectMemory>>formatOf: defines the class format bits in the header.
>
> Yes we saw that.
>
>> Or do you mean the 3 header formats? Read the class comment of ObjectMemory.
>>
>> Also, there is a nice explanation in section A.1 of Tim's chapter in the NuBlue book:
>>
>> http://www.rowledge.org/tim/squeak/assets/OE-Tour.pdf
>
> Another question was where can we get a description of the image format.
> I was planning to read the VM C code included the generated one.

The image is just a header plus a memory dump of the object memory. The header is written in Interpreter>>writeImageFileIO: and read in Interpreter>>readImageFromFile:HeapSize:StartingAt:.

- Bert -

best
Eliot 
Reply | Threaded
Open this post in threaded view
|

Re: About the image

Andreas.Raab
 
On 8/17/2010 6:24 PM, Eliot Miranda wrote:

> On Tue, Aug 17, 2010 at 5:38 AM, Bert Freudenberg <[hidden email]
> <mailto:[hidden email]>> wrote:
>     I thought the three inst vars in Behavior was all the VM knew and
>     cared about? I'd consider everything else a bug ;)
>
>
> That's right.  See all implementors of initializeClassIndices:
>
> BTW, you /can/ add an inst var to Behavior.  Try and evaluate
>
> Object subclass: #Behavior
> instanceVariableNames: 'superclass methodDict format extra'
> classVariableNames: 'ObsoleteSubclasses'
> poolDictionaries: ''
> category: 'Kernel-Classes'
>
> Works for me in a 4.1 derived image.

The problem is in Interpreter>>classNameOf: aClass Is: className which
uses the Class' name ivar directly. This causes several "interesting"
issues with the SmartSyntaxBlaBlaPlugins (such as JPEGReadWriterPlugin).

Cheers,
   - Andreas
Reply | Threaded
Open this post in threaded view
|

Re: About the image

Eliot Miranda-2
 


On Tue, Aug 17, 2010 at 6:41 PM, Andreas Raab <[hidden email]> wrote:

On 8/17/2010 6:24 PM, Eliot Miranda wrote:
On Tue, Aug 17, 2010 at 5:38 AM, Bert Freudenberg <[hidden email]
<mailto:[hidden email]>> wrote:
   I thought the three inst vars in Behavior was all the VM knew and
   cared about? I'd consider everything else a bug ;)


That's right.  See all implementors of initializeClassIndices:

BTW, you /can/ add an inst var to Behavior.  Try and evaluate

Object subclass: #Behavior
instanceVariableNames: 'superclass methodDict format extra'
classVariableNames: 'ObsoleteSubclasses'
poolDictionaries: ''
category: 'Kernel-Classes'

Works for me in a 4.1 derived image.

The problem is in Interpreter>>classNameOf: aClass Is: className which uses the Class' name ivar directly. This causes several "interesting" issues with the SmartSyntaxBlaBlaPlugins (such as JPEGReadWriterPlugin).

Eugh.  In the VW VM I added code to choose a class in the specialObjectsArray (Array) and search it for a byte object with the characters 'Array' and use that as the class name index.  Similarly its class was searched for an object equal to Array and that slot was used as the thisClass index.  I see I was too lazy to do that in StackInterpreter>>initializeInterpreter but I should, and should introduce classNameIndex and thisClassIndex and use them in place of the hard-coded constants.  That way one can save immediately after adding/removing the behavior inst var and have the VM get things right on start-up.


Cheers,
 - Andreas

Reply | Threaded
Open this post in threaded view
|

Re: About the image

Andreas.Raab
 
On 8/17/2010 6:54 PM, Eliot Miranda wrote:

> In the VW VM I added code to choose a class in the
> specialObjectsArray (Array) and search it for a byte object with the
> characters 'Array' and use that as the class name index.  Similarly its
> class was searched for an object equal to Array and that slot was used
> as the thisClass index.  I see I was too lazy to do that in
> StackInterpreter>>initializeInterpreter but I should, and should
> introduce classNameIndex and thisClassIndex and use them in place of the
> hard-coded constants.  That way one can save immediately after
> adding/removing the behavior inst var and have the VM get things right
> on start-up.

That's one option. An alternative is to provide the class name index via
the splObjsArray and default to 5 if not present.

Cheers,
   - Andreas
Reply | Threaded
Open this post in threaded view
|

Re: About the image

Bert Freudenberg


On 18.08.2010, at 04:48, Andreas Raab wrote:

> On 8/17/2010 6:54 PM, Eliot Miranda wrote:
>> In the VW VM I added code to choose a class in the
>> specialObjectsArray (Array) and search it for a byte object with the
>> characters 'Array' and use that as the class name index.  Similarly its
>> class was searched for an object equal to Array and that slot was used
>> as the thisClass index.  I see I was too lazy to do that in
>> StackInterpreter>>initializeInterpreter but I should, and should
>> introduce classNameIndex and thisClassIndex and use them in place of the
>> hard-coded constants.  That way one can save immediately after
>> adding/removing the behavior inst var and have the VM get things right
>> on start-up.
>
> That's one option. An alternative is to provide the class name index via the splObjsArray and default to 5 if not present.

I like Elliot's option better. Not having to maintain that index means there is one thing less to worry about. If the added flexibility of an explicit index is needed later, it could still be added at that point.

- Bert -

Reply | Threaded
Open this post in threaded view
|

Re: About the image

David T. Lewis
In reply to this post by Eliot Miranda-2
 
On Tue, Aug 17, 2010 at 06:24:40PM -0700, Eliot Miranda wrote:
>
> BTW St?phane,  IMO the VM's Slang code is far friendlier than the generated
> C.  The Slang translator is the thing that's unfriendly.  It doesn't do much
> verification and so when it mistranslates it can be hard to figure out
> what's going on.  but the Slang/Smalltalk that implements the VM is fine.
>

Yes, and it is also much easier to read and understand the Slang/Smalltalk
directly in the image.

It is also to be able to look at the C code, but you can do this directly
in the image with SlangBrowser. In Pharo this is not integrated into the
browsers (maybe someone can supply a patch for this?), but you can still
display e.g. the generated C code for ObjectMemory>>fullGC from a
workspace doit:

  Transcript clear; show: (ObjectMemory asCString: #fullGC)

Or the fully inlined version as it would appear in the interp.c file:

  Transcript clear; show: (ObjectMemory asInlinedCString: #fullGC)

Dave

Reply | Threaded
Open this post in threaded view
|

Re: About the image

Andreas.Raab
In reply to this post by Bert Freudenberg
 
On 8/18/2010 2:21 AM, Bert Freudenberg wrote:

> On 18.08.2010, at 04:48, Andreas Raab wrote:
>> On 8/17/2010 6:54 PM, Eliot Miranda wrote:
>>> In the VW VM I added code to choose a class in the
>>> specialObjectsArray (Array) and search it for a byte object with the
>>> characters 'Array' and use that as the class name index.  Similarly its
>>> class was searched for an object equal to Array and that slot was used
>>> as the thisClass index.  I see I was too lazy to do that in
>>> StackInterpreter>>initializeInterpreter but I should, and should
>>> introduce classNameIndex and thisClassIndex and use them in place of the
>>> hard-coded constants.  That way one can save immediately after
>>> adding/removing the behavior inst var and have the VM get things right
>>> on start-up.
>>
>> That's one option. An alternative is to provide the class name index via the splObjsArray and default to 5 if not present.
>
> I like Elliot's option better. Not having to maintain that index means there is one thing less to worry about. If the added flexibility of an explicit index is needed later, it could still be added at that point.

I'm not entirely sure about that. You do have to maintain the
information that after adding an ivar you'll have to save and restart
the image or else strange things will happen. This effect is nowhere
documented. With the index you could automate the process (using the
ClassBuilder notification) so that if Class chnages shape it updates the
splObj entry automatically.

Cheers,
   - Andreas
Reply | Threaded
Open this post in threaded view
|

Re: About the image

Eliot Miranda-2
 


On Wed, Aug 18, 2010 at 8:33 AM, Andreas Raab <[hidden email]> wrote:

On 8/18/2010 2:21 AM, Bert Freudenberg wrote:
On 18.08.2010, at 04:48, Andreas Raab wrote:
On 8/17/2010 6:54 PM, Eliot Miranda wrote:
In the VW VM I added code to choose a class in the
specialObjectsArray (Array) and search it for a byte object with the
characters 'Array' and use that as the class name index.  Similarly its
class was searched for an object equal to Array and that slot was used
as the thisClass index.  I see I was too lazy to do that in
StackInterpreter>>initializeInterpreter but I should, and should
introduce classNameIndex and thisClassIndex and use them in place of the
hard-coded constants.  That way one can save immediately after
adding/removing the behavior inst var and have the VM get things right
on start-up.

That's one option. An alternative is to provide the class name index via the splObjsArray and default to 5 if not present.

I like Elliot's option better. Not having to maintain that index means there is one thing less to worry about. If the added flexibility of an explicit index is needed later, it could still be added at that point.

I'm not entirely sure about that. You do have to maintain the information that after adding an ivar you'll have to save and restart the image or else strange things will happen. This effect is nowhere documented. With the index you could automate the process (using the ClassBuilder notification) so that if Class chnages shape it updates the splObj entry automatically.

That's a fair point, but changing the shape of Behavior is an edge case anyway, and the VM code with the specialObjectsArray approach is ugly
    self integerValueFor: (self splObj: ClassNameIndexIndex)
and much more fragile (e.g. if this entry gets damaged then no printCallStack)

One could just as easily document the VM behavior by changing the ClassBuilder to notify that when the name and thisClass inst vars change their index the programmer should save quit and resume, as make the ClassBuilder update a specialObjectsArray entry.

best
Eliot


Cheers,
 - Andreas

Reply | Threaded
Open this post in threaded view
|

Re: About the image

K K Subbu
In reply to this post by stephane ducasse-2
 
On Tuesday 17 Aug 2010 4:57:32 pm stephane ducasse wrote:
> Another question was where can we get a description of the image format.
> I was planning to read the VM C code included the generated one.
I have attached the notes and an image dump program I wrote when I was
studying the image from its bits.

HTH .. Subbu

vmnotes.txt (15K) Download Attachment
sq.c (5K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: About the image

Andreas.Raab
In reply to this post by Eliot Miranda-2
 
On 8/18/2010 9:59 AM, Eliot Miranda wrote:

> On Wed, Aug 18, 2010 at 8:33 AM, Andreas Raab <[hidden email]
> <mailto:[hidden email]>> wrote:
>     I'm not entirely sure about that. You do have to maintain the
>     information that after adding an ivar you'll have to save and
>     restart the image or else strange things will happen. This effect is
>     nowhere documented. With the index you could automate the process
>     (using the ClassBuilder notification) so that if Class chnages shape
>     it updates the splObj entry automatically.
>
> That's a fair point, but changing the shape of Behavior is an edge case
> anyway, and the VM code with the specialObjectsArray approach is ugly
>      self integerValueFor: (self splObj: ClassNameIndexIndex)
> and much more fragile (e.g. if this entry gets damaged then no
> printCallStack)
>
> One could just as easily document the VM behavior by changing the
> ClassBuilder to notify that when the name and thisClass inst vars change
> their index the programmer should save quit and resume, as make the
> ClassBuilder update a specialObjectsArray entry.

True. In reality, I don't care which way it's done but I agree with
Bert's fundamental point that having hidden information about that stuff
ain't a good idea so having such notification sounds like a very good
idea indeed (and very easy to do).

Cheers,
   - Andreas
Reply | Threaded
Open this post in threaded view
|

Re: About the image

stephane ducasse-2
In reply to this post by Eliot Miranda-2

>
> The problem is in Interpreter>>classNameOf: aClass Is: className which uses the Class' name ivar directly. This causes several "interesting" issues with the SmartSyntaxBlaBlaPlugins (such as JPEGReadWriterPlugin).

Yes marcus was reminding that.

> Eugh.  In the VW VM I added code to choose a class in the specialObjectsArray (Array) and search it for a byte object with the characters 'Array' and use that as the class name index.  Similarly its class was searched for an object equal to Array and that slot was used as the thisClass index.  I see I was too lazy to do that in StackInterpreter>>initializeInterpreter but I should, and should introduce classNameIndex and thisClassIndex and use them in place of the hard-coded constants.  That way one can save immediately after adding/removing the behavior inst var and have the VM get things right on start-up.

late binding decision is what I like :)

Stef
Reply | Threaded
Open this post in threaded view
|

Re: About the image

stephane ducasse-2
In reply to this post by K K Subbu
 
Thanks!

Stef

On Aug 19, 2010, at 3:51 AM, K. K. Subramaniam wrote:

> On Tuesday 17 Aug 2010 4:57:32 pm stephane ducasse wrote:
>> Another question was where can we get a description of the image format.
>> I was planning to read the VM C code included the generated one.
> I have attached the notes and an image dump program I wrote when I was
> studying the image from its bits.
>
> HTH .. Subbu
> <vmnotes.txt><sq.c>

Reply | Threaded
Open this post in threaded view
|

Re: About the image

K K Subbu
In reply to this post by K K Subbu
 
On Thursday 19 Aug 2010 6:45:17 pm you wrote:
> do you mind to release a version under MIT/BSD?
Not at all. Attached is the new source code.

Subbu