Re: [squeak-dev] uncompacting classes

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] uncompacting classes

Eliot Miranda-2
 


On Sat, Nov 26, 2011 at 3:02 PM, Chris Muller <[hidden email]> wrote:
Compact classes cannot be made uncompact in a Cog+JIT VM.

Not exactly true.  Certain classes cannot be uncompacted.  These are as defined by
    StackInterpreter>>#checkAssumedCompactClasses
and the ones that can't be uncompacted are
Array
LargeNegativeInteger
LargePositiveInteger
Float
MethodContext

There is a performance advantage to being able to identify instances of these classes from the compact class index.

But any other classes should be able to be compacted and uncompacted.


Can compact classes be made uncompact when running the StackInterpreter VM?

It is exactly the same story.  The same classes are assumed to be compact in the StackInterpreter VM as the CoInterpreter VM.


Thanks.



HTH
--
best,
Eliot

Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] uncompacting classes

Mariano Martinez Peck
 


On Mon, Nov 28, 2011 at 7:25 PM, Eliot Miranda <[hidden email]> wrote:
 


On Sat, Nov 26, 2011 at 3:02 PM, Chris Muller <[hidden email]> wrote:
Compact classes cannot be made uncompact in a Cog+JIT VM.

Not exactly true.  Certain classes cannot be uncompacted.  These are as defined by
    StackInterpreter>>#checkAssumedCompactClasses
and the ones that can't be uncompacted are
Array
LargeNegativeInteger
LargePositiveInteger
Float
MethodContext

There is a performance advantage to being able to identify instances of these classes from the compact class index.

But any other classes should be able to be compacted and uncompacted.


Eliot, should we validate this in image side (#becomeUncompact)  ?

 


Can compact classes be made uncompact when running the StackInterpreter VM?

It is exactly the same story.  The same classes are assumed to be compact in the StackInterpreter VM as the CoInterpreter VM.


Thanks.



HTH
--
best,
Eliot





--
Mariano
http://marianopeck.wordpress.com

Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] uncompacting classes

Eliot Miranda-2
 


On Tue, Nov 29, 2011 at 11:54 AM, Mariano Martinez Peck <[hidden email]> wrote:
 


On Mon, Nov 28, 2011 at 7:25 PM, Eliot Miranda <[hidden email]> wrote:
 


On Sat, Nov 26, 2011 at 3:02 PM, Chris Muller <[hidden email]> wrote:
Compact classes cannot be made uncompact in a Cog+JIT VM.

Not exactly true.  Certain classes cannot be uncompacted.  These are as defined by
    StackInterpreter>>#checkAssumedCompactClasses
and the ones that can't be uncompacted are
Array
LargeNegativeInteger
LargePositiveInteger
Float
MethodContext

There is a performance advantage to being able to identify instances of these classes from the compact class index.

But any other classes should be able to be compacted and uncompacted.


Eliot, should we validate this in image side (#becomeUncompact)  ?

I suppose so.  The "right" way to do this would be to ask the VM (via a primitive) for the set of assumed compact classes, but that's too much work.  I hope that a new GC/object representation will become available before I would ever think of changing the set of compact classes, so having the method document what the current VM requires is ok.


 


Can compact classes be made uncompact when running the StackInterpreter VM?

It is exactly the same story.  The same classes are assumed to be compact in the StackInterpreter VM as the CoInterpreter VM.


Thanks.



HTH
--
best,
Eliot





--
Mariano
http://marianopeck.wordpress.com





--
best,
Eliot

Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] uncompacting classes

Mariano Martinez Peck
 


On Tue, Nov 29, 2011 at 10:27 PM, Eliot Miranda <[hidden email]> wrote:
 


On Tue, Nov 29, 2011 at 11:54 AM, Mariano Martinez Peck <[hidden email]> wrote:
 


On Mon, Nov 28, 2011 at 7:25 PM, Eliot Miranda <[hidden email]> wrote:
 


On Sat, Nov 26, 2011 at 3:02 PM, Chris Muller <[hidden email]> wrote:
Compact classes cannot be made uncompact in a Cog+JIT VM.

Not exactly true.  Certain classes cannot be uncompacted.  These are as defined by
    StackInterpreter>>#checkAssumedCompactClasses
and the ones that can't be uncompacted are
Array
LargeNegativeInteger
LargePositiveInteger
Float
MethodContext

There is a performance advantage to being able to identify instances of these classes from the compact class index.

But any other classes should be able to be compacted and uncompacted.


Eliot, should we validate this in image side (#becomeUncompact)  ?

I suppose so.  The "right" way to do this would be to ask the VM (via a primitive) for the set of assumed compact classes, but that's too much work.

I agree. So I will try to add the validation.
 
 I hope that a new GC/object representation will become available before I would ever think of changing the set of compact classes, so having the method document what the current VM requires is ok.

Eliot, a simple question: In Pharo:
Smalltalk compactClassesArray asSet size -> 15
Smalltalk compactClassesArray asSet size -> 13

I would like to have one extra free bit in the object header. I can hack my own VM which uses 4 bits for CompactClasses rather than 5, but do you think we can do this for the official Cog VM as well? this would allow "researched" a much nice infrastructure out of the box. How much work can be such change? is there someone needing 32 compact classes?  if I do a SpaceTally new printSpaceAnalysis  it looks like if I only need the first 10 classes....

I know in the future you want to change all this thing about compact classes, but if we can have one free bit tomorrow (instead of "in the future"), then this is very very good.

Thanks in advance,


 


 


Can compact classes be made uncompact when running the StackInterpreter VM?

It is exactly the same story.  The same classes are assumed to be compact in the StackInterpreter VM as the CoInterpreter VM.


Thanks.



HTH
--
best,
Eliot





--
Mariano
http://marianopeck.wordpress.com





--
best,
Eliot





--
Mariano
http://marianopeck.wordpress.com

Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] uncompacting classes

Stefan Marr

Hi:

On 30 Nov 2011, at 10:19, Mariano Martinez Peck wrote:

> I would like to have one extra free bit in the object header. I can hack my own VM which uses 4 bits for CompactClasses rather than 5, but do you think we can do this for the official Cog VM as well? this would allow "researched" a much nice infrastructure out of the box.

If you really need more flexibility, consider adding a compile-time option to add an extra preheader to each object.
We have that in the RoarVM.
Of course, there is a performance impact. But in return you get what you have dreamed of in your wildest dreams... And it is easy to add just another word, or another one...

We had a bug recently, that required me to clean up the code, I could point you at the fix. That should give an indication where you would need to adapt the VM.
And this approach should work at least for the classic interpreter. Not sure what that would entail for the stackVM, but for Cog you would probably need to adapt the code generation.

Best regards
Stefan


--
Stefan Marr
Software Languages Lab
Vrije Universiteit Brussel
Pleinlaan 2 / B-1050 Brussels / Belgium
http://soft.vub.ac.be/~smarr
Phone: +32 2 629 2974
Fax:   +32 2 629 3525

Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] uncompacting classes

Eliot Miranda-2
In reply to this post by Mariano Martinez Peck
 


On Wed, Nov 30, 2011 at 1:19 AM, Mariano Martinez Peck <[hidden email]> wrote:
 


On Tue, Nov 29, 2011 at 10:27 PM, Eliot Miranda <[hidden email]> wrote:
 


On Tue, Nov 29, 2011 at 11:54 AM, Mariano Martinez Peck <[hidden email]> wrote:
 


On Mon, Nov 28, 2011 at 7:25 PM, Eliot Miranda <[hidden email]> wrote:
 


On Sat, Nov 26, 2011 at 3:02 PM, Chris Muller <[hidden email]> wrote:
Compact classes cannot be made uncompact in a Cog+JIT VM.

Not exactly true.  Certain classes cannot be uncompacted.  These are as defined by
    StackInterpreter>>#checkAssumedCompactClasses
and the ones that can't be uncompacted are
Array
LargeNegativeInteger
LargePositiveInteger
Float
MethodContext

There is a performance advantage to being able to identify instances of these classes from the compact class index.

But any other classes should be able to be compacted and uncompacted.


Eliot, should we validate this in image side (#becomeUncompact)  ?

I suppose so.  The "right" way to do this would be to ask the VM (via a primitive) for the set of assumed compact classes, but that's too much work.

I agree. So I will try to add the validation.
 
 I hope that a new GC/object representation will become available before I would ever think of changing the set of compact classes, so having the method document what the current VM requires is ok.

Eliot, a simple question: In Pharo:
Smalltalk compactClassesArray asSet size -> 15
Smalltalk compactClassesArray asSet size -> 13

I would like to have one extra free bit in the object header. I can hack my own VM which uses 4 bits for CompactClasses rather than 5, but do you think we can do this for the official Cog VM as well? this would allow "researched" a much nice infrastructure out of the box. How much work can be such change? is there someone needing 32 compact classes?  if I do a SpaceTally new printSpaceAnalysis  it looks like if I only need the first 10 classes....

I know in the future you want to change all this thing about compact classes, but if we can have one free bit tomorrow (instead of "in the future"), then this is very very good.

Seems reasonable.  What do you think Andreas, David, Esteban, Ian?  Shall we make this change?
 

Thanks in advance,


 


 


Can compact classes be made uncompact when running the StackInterpreter VM?

It is exactly the same story.  The same classes are assumed to be compact in the StackInterpreter VM as the CoInterpreter VM.


Thanks.



HTH
--
best,
Eliot





--
Mariano
http://marianopeck.wordpress.com





--
best,
Eliot





--
Mariano
http://marianopeck.wordpress.com





--
best,
Eliot

Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] uncompacting classes

Eliot Miranda-2
 


On Wed, Nov 30, 2011 at 7:39 PM, Eliot Miranda <[hidden email]> wrote:


On Wed, Nov 30, 2011 at 1:19 AM, Mariano Martinez Peck <[hidden email]> wrote:
 


On Tue, Nov 29, 2011 at 10:27 PM, Eliot Miranda <[hidden email]> wrote:
 


On Tue, Nov 29, 2011 at 11:54 AM, Mariano Martinez Peck <[hidden email]> wrote:
 


On Mon, Nov 28, 2011 at 7:25 PM, Eliot Miranda <[hidden email]> wrote:
 


On Sat, Nov 26, 2011 at 3:02 PM, Chris Muller <[hidden email]> wrote:
Compact classes cannot be made uncompact in a Cog+JIT VM.

Not exactly true.  Certain classes cannot be uncompacted.  These are as defined by
    StackInterpreter>>#checkAssumedCompactClasses
and the ones that can't be uncompacted are
Array
LargeNegativeInteger
LargePositiveInteger
Float
MethodContext

There is a performance advantage to being able to identify instances of these classes from the compact class index.

But any other classes should be able to be compacted and uncompacted.


Eliot, should we validate this in image side (#becomeUncompact)  ?

I suppose so.  The "right" way to do this would be to ask the VM (via a primitive) for the set of assumed compact classes, but that's too much work.

I agree. So I will try to add the validation.
 
 I hope that a new GC/object representation will become available before I would ever think of changing the set of compact classes, so having the method document what the current VM requires is ok.

Eliot, a simple question: In Pharo:
Smalltalk compactClassesArray asSet size -> 15
Smalltalk compactClassesArray asSet size -> 13

I would like to have one extra free bit in the object header. I can hack my own VM which uses 4 bits for CompactClasses rather than 5, but do you think we can do this for the official Cog VM as well? this would allow "researched" a much nice infrastructure out of the box. How much work can be such change? is there someone needing 32 compact classes?  if I do a SpaceTally new printSpaceAnalysis  it looks like if I only need the first 10 classes....

I know in the future you want to change all this thing about compact classes, but if we can have one free bit tomorrow (instead of "in the future"), then this is very very good.

Seems reasonable.  What do you think Andreas, David, Esteban, Ian?  Shall we make this change?

Belay that.  It doesn't fly.  Bitmap is at index 16, not 15.  So close.  Ah well...  Mariano, you could perhaps make it an option.
 
 

Thanks in advance,


 


 


Can compact classes be made uncompact when running the StackInterpreter VM?

It is exactly the same story.  The same classes are assumed to be compact in the StackInterpreter VM as the CoInterpreter VM.


Thanks.



HTH
--
best,
Eliot





--
Mariano
http://marianopeck.wordpress.com





--
best,
Eliot





--
Mariano
http://marianopeck.wordpress.com





--
best,
Eliot




--
best,
Eliot

Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] uncompacting classes

Mariano Martinez Peck
 


Eliot, a simple question: In Pharo:
Smalltalk compactClassesArray asSet size -> 15
Smalltalk compactClassesArray asSet size -> 13

I would like to have one extra free bit in the object header. I can hack my own VM which uses 4 bits for CompactClasses rather than 5, but do you think we can do this for the official Cog VM as well? this would allow "researched" a much nice infrastructure out of the box. How much work can be such change? is there someone needing 32 compact classes?  if I do a SpaceTally new printSpaceAnalysis  it looks like if I only need the first 10 classes....

I know in the future you want to change all this thing about compact classes, but if we can have one free bit tomorrow (instead of "in the future"), then this is very very good.

Seems reasonable.  What do you think Andreas, David, Esteban, Ian?  Shall we make this change?

Belay that.  It doesn't fly.  Bitmap is at index 16, not 15.  So close.  Ah well...  Mariano, you could perhaps make it an option.
 

Hi Eliot. Just to understand, what is the problem that Bitmap index is 16? It is not possible to uncompact and compact again or something like this to assign Bitmap the next "free index" ?   Bitmap is not in #checkAssumedCompactClasses

Thanks

 
 

Thanks in advance,


 


 


Can compact classes be made uncompact when running the StackInterpreter VM?

It is exactly the same story.  The same classes are assumed to be compact in the StackInterpreter VM as the CoInterpreter VM.


Thanks.



HTH
--
best,
Eliot





--
Mariano
http://marianopeck.wordpress.com





--
best,
Eliot





--
Mariano
http://marianopeck.wordpress.com





--
best,
Eliot




--
best,
Eliot





--
Mariano
http://marianopeck.wordpress.com

Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] uncompacting classes

Mariano Martinez Peck
In reply to this post by Stefan Marr
 


On Wed, Nov 30, 2011 at 10:47 AM, Stefan Marr <[hidden email]> wrote:

Hi:

On 30 Nov 2011, at 10:19, Mariano Martinez Peck wrote:

> I would like to have one extra free bit in the object header. I can hack my own VM which uses 4 bits for CompactClasses rather than 5, but do you think we can do this for the official Cog VM as well? this would allow "researched" a much nice infrastructure out of the box.

If you really need more flexibility, consider adding a compile-time option to add an extra preheader to each object.
We have that in the RoarVM.
Of course, there is a performance impact. But in return you get what you have dreamed of in your wildest dreams... And it is easy to add just another word, or another one...


That' would be very nice to have :)
 
We had a bug recently, that required me to clean up the code, I could point you at the fix. That should give an indication where you would need to adapt the VM.
And this approach should work at least for the classic interpreter. Not sure what that would entail for the stackVM, but for Cog you would probably need to adapt the code generation.

Best regards
Stefan


--
Stefan Marr
Software Languages Lab
Vrije Universiteit Brussel
Pleinlaan 2 / B-1050 Brussels / Belgium
http://soft.vub.ac.be/~smarr
Phone: <a href="tel:%2B32%202%20629%202974" value="+3226292974">+32 2 629 2974
Fax:   <a href="tel:%2B32%202%20629%203525" value="+3226293525">+32 2 629 3525




--
Mariano
http://marianopeck.wordpress.com

Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] uncompacting classes

Stefan Marr


On 01 Dec 2011, at 10:03, Mariano Martinez Peck wrote:

> That' would be very nice to have :)
>

Well, it is a bit of work, I suppose.
You will need to adapt the image reading and spread them wider into your memory, adding that extra space in front of every object.

That also means you'll have to correct all the pointers in the objects.
That's all kind of trivial with an object table you haven't got, I think.
Perhaps, someone has code for that lying around on a dusty shelf?

Afterwards, if I remember correctly, it is mostly heap traversal that needs to be adapted.

We have all pointers still pointing to the real objects, and only the GC and traversal routines do know about the preheader. They just skip it. (Depending on what you want to put into the header, you might want to make the GC aware of that. I actually keep pointer there, and I think, that is currently not taken into account by the GC. oops...)

From there, you can just access it at a negative offset, almost like a normal object field (be careful with the variable width object header).

Here some hints in our code:

https://github.com/smarr/RoarVM/commit/2750484e5cf29248390d8c56a2a250a30e899fa1
https://github.com/smarr/RoarVM/commit/5dcd6a1662d87a7d06a934957b1f2cce90778e98

Best regards
Stefan


--
Stefan Marr
Software Languages Lab
Vrije Universiteit Brussel
Pleinlaan 2 / B-1050 Brussels / Belgium
http://soft.vub.ac.be/~smarr
Phone: +32 2 629 2974
Fax:   +32 2 629 3525

Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] uncompacting classes

David T. Lewis
In reply to this post by Eliot Miranda-2
 
On Wed, Nov 30, 2011 at 07:39:05PM -0800, Eliot Miranda wrote:

> On Wed, Nov 30, 2011 at 1:19 AM, Mariano Martinez Peck <
> [hidden email]> wrote:
> >
> > Eliot, a simple question: In Pharo:
> > Smalltalk compactClassesArray asSet size -> 15
> > Smalltalk compactClassesArray asSet size -> 13
> >
> > I would like to have one extra free bit in the object header. I can hack
> > my own VM which uses 4 bits for CompactClasses rather than 5, but do you
> > think we can do this for the official Cog VM as well? this would allow
> > "researched" a much nice infrastructure out of the box. How much work can
> > be such change? is there someone needing 32 compact classes?  if I do a
> > SpaceTally new printSpaceAnalysis  it looks like if I only need the first
> > 10 classes....
> >
> > I know in the future you want to change all this thing about compact
> > classes, but if we can have one free bit tomorrow (instead of "in the
> > future"), then this is very very good.
> >
>
> Seems reasonable.  What do you think Andreas, David, Esteban, Ian?  Shall
> we make this change?

My answer is strictly from the point of view of VM maintainance. As long
as we manage the change so as to not cause problems for existing VMs and
images, then this proposal has my full support.

By "manage the change" I mean:

- Agree in advance a new image format number (i.e. allocate a free bit
in the image format number) such that a VM can decide based on the image
format whether or not it knows how to support this format.

- Document the change in the ImageFormat package in VMMaker, such that
the new image format identification will be documented to avoid conflicts
with other VM projects.

- Keep the ObjectMemory related code organized such that we can easily
generate both the old and new format.

Eliot, I think that the way you have separated ObjectMemory from Interpreter
in VMMaker makes this sort of change very easy to manage, so from my
point of view if you think this is a good thing to do, there is no reason
not to do it.

Dave