Fwd: [Pharo-project] Could not load VMMaker in 10315

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Fwd: [Pharo-project] Could not load VMMaker in 10315

johnmci



Begin forwarded message:

> From: Stéphane Ducasse <[hidden email]>
> Date: May 24, 2009 1:35:01 AM PDT (CA)
> To: Pharo Development <[hidden email]>
> Subject: [Pharo-project] Could not load VMMaker in 10315
> Reply-To: [hidden email]
>
> Hi
>
> I did
>
> ScriptLoader loadFFI
> Then loaded VMMaker-dtl.122
> Then proceeded when asked about klatt.
>
> And I got Cannot compile -- stack including temps is too deep
>
> what did I miss?
>
> Stef
>
> _______________________________________________
> Pharo-project mailing list
> [hidden email]
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

--
=
=
=
========================================================================
John M. McIntosh <[hidden email]>   Twitter:  
squeaker68882
Corporate Smalltalk Consulting Ltd.  http://www.smalltalkconsulting.com
=
=
=
========================================================================




Reply | Threaded
Open this post in threaded view
|

Re: Fwd: [Pharo-project] Could not load VMMaker in 10315

Eliot Miranda-2
 


On Sun, May 24, 2009 at 1:13 PM, John M McIntosh <[hidden email]> wrote:



Begin forwarded message:

From: Stéphane Ducasse <[hidden email]>
Date: May 24, 2009 1:35:01 AM PDT (CA)
To: Pharo Development <[hidden email]>
Subject: [Pharo-project] Could not load VMMaker in 10315
Reply-To: [hidden email]

Hi

I did

ScriptLoader loadFFI
Then loaded VMMaker-dtl.122
Then proceeded when asked about klatt.

And I got Cannot compile -- stack including temps is too deep

what did I miss?

Nothing.  There is a method in the GeniePlugin who's stack is too deep to compile with closures.  Here are some workarounds and solutions in increasing levels of difficulty:

1. since the GeniePlugin is inessential simply move it to another package and ignore the problem (this is what we have done at Qwaq).

2a. Since the method is never executed one can temporarily change the stack limit (e.g. CompiledMethod classPool at: #LargeFrame put: 57) and load the package, reverting the stack limit after loading (Compiledmethod classPool at: #LargeFrame put: 56)

2b. change CompiledMethod>>needsFrame: so that proceeding from the error still creates a compiled method (i.e. remove the ^ return from the method so that it reads
        needsFrameSize: newFrameSize 
                 
"Set the largeFrameBit to accomodate the newFrameSize" 
                 
| largeFrameBit header | 
                 
largeFrameBit := 16r20000
                 (
self numTemps + newFrameSize> LargeFrame ifTrue: 
                          [
self error: 'Cannot compile -- stack including temps is too deep']. 
                 
header := self objectAt: 1
                 (
header bitAnd: largeFrameBit~= 0 
                          
ifTrue: [header := header - largeFrameBit]. 
                 
self objectAt: 1 put: header 
                                   
+ ((self numTemps + newFrameSize) > SmallFrame 
                                                     
ifTrue: [largeFrameBit] 
                                                     
ifFalse: [0])

then file-in the method and proceed through the warning.
Both of these are tedious but since you're probably not going to be changing the method and since Monticello won't recompile it once it is loaded it'll sit there quite happily once compiled.

3. refactor the GeniePlugin method into an outer args parser and an inner engine method and solve the problem.  Since VMMaker will inline anyway this shouldn't make performance worse.  I haven't done this because I'm not the author and it is quite a complex method.

4. modify the Closure compiler so that it reuses temporaries in optimized blocks.  The method compiles with the BlueBook compiler because in
1 to: n do: [:i| .......].
...
1 to: n do: [:i| .......].
the two block variables "i" are the same "physical" variable stored in the same location in the home context (because BlueBook blocks do not have local arguments or temporaries).
But in the Closure compiler they are logically distinct variables and if the blocks were real and not optimized they would occupy different "physical" locations.  Since they are optimized blocks their temporaries do get allocated in the home context, but they occupy different locations, and hence the stack size is larger in the Closure compiler.  The Closure compiler could be modified to use a more sophisticated temporary variable scope analysis that would allow it to determine that the first block's "i" is out of scope when the second block is compiled and hence that the location can be reused.
This is non-trivial because it requires rewriting much of the temporary scope management (see TempVariableNode>>scope[:] & senders of scope:).  If I had hit the stack limit in more than this one method I probably would have tried to solve this but for now there are fatter fish to fry.


I plan that eventually the Cog VM will support a CompiledMethod format that will have a higher argument, temp and total frame size limit, but this isn't a priority right now.

Apologies
Eliot



Stef

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

--
=
=
=
========================================================================
John M. McIntosh <[hidden email]>   Twitter:  squeaker68882
Corporate Smalltalk Consulting Ltd.  http://www.smalltalkconsulting.com
===========================================================================





Reply | Threaded
Open this post in threaded view
|

Re: Fwd: [Pharo-project] Could not load VMMaker in 10315

David T. Lewis
 
On Sun, May 24, 2009 at 01:44:52PM -0700, Eliot Miranda wrote:
>
> 3. refactor the GeniePlugin method into an outer args parser and an inner
> engine method and solve the problem.  Since VMMaker will inline anyway this
> shouldn't make performance worse.  I haven't done this because I'm not the
> author and it is quite a complex method.

This sounds like the right thing to do, regardless of what may change
in the compiler(s).

Dave

Reply | Threaded
Open this post in threaded view
|

Re: Fwd: [Pharo-project] Could not load VMMaker in 10315

Eliot Miranda-2
 


On Mon, May 25, 2009 at 7:40 AM, David T. Lewis <[hidden email]> wrote:

On Sun, May 24, 2009 at 01:44:52PM -0700, Eliot Miranda wrote:
>
> 3. refactor the GeniePlugin method into an outer args parser and an inner
> engine method and solve the problem.  Since VMMaker will inline anyway this
> shouldn't make performance worse.  I haven't done this because I'm not the
> author and it is quite a complex method.

This sounds like the right thing to do, regardless of what may change
in the compiler(s).

+1
 


Dave


Reply | Threaded
Open this post in threaded view
|

Re: Fwd: [Pharo-project] Could not load VMMaker in 10315

David T. Lewis
 
On Mon, May 25, 2009 at 11:06:10AM -0700, Eliot Miranda wrote:

>  
> On Mon, May 25, 2009 at 7:40 AM, David T. Lewis <[hidden email]> wrote:
>
> >
> > On Sun, May 24, 2009 at 01:44:52PM -0700, Eliot Miranda wrote:
> > >
> > > 3. refactor the GeniePlugin method into an outer args parser and an inner
> > > engine method and solve the problem.  Since VMMaker will inline anyway
> > this
> > > shouldn't make performance worse.  I haven't done this because I'm not
> > the
> > > author and it is quite a complex method.
> >
> > This sounds like the right thing to do, regardless of what may change
> > in the compiler(s).
>
>
> +1
>

Well unfortunately the refactoring turns out to be easier said than done.
At least it was easier for me to say it than it was for me to do it.

I made a naive attempt to split the primitive into smaller methods, but this
just ended up trading off too many things on the stack for too many arguments
in an argument list.

The method temp variables could be changed to be instance variables. This
would probably work fine for any current VMs (not sure about Hydra), but
it does not pass the smell test IMO.

I suspect that Nathaniel Schaerli knew what he was doing when he wrote
this primitive, and I most certainly do not, so I don't expect that I will
make much further progress on this front.

Dave

Reply | Threaded
Open this post in threaded view
|

Re: Fwd: [Pharo-project] Could not load VMMaker in 10315

David T. Lewis
 
VMMaker (from SqueakSource) can now be loaded into closure-enabled images.

I was able to remove a couple of temp variables from the GeniePlugin
primitive without affecting the logic. This was discussed on several
lists back in May.

The fix is in VMMaker-dtl.135 on SqueakSource, and is documented in
Mantis http://bugs.squeak.org/view.php?id=7384.

Dave

On Mon, May 25, 2009 at 10:37:15PM -0400, David T. Lewis wrote:

>  
> On Mon, May 25, 2009 at 11:06:10AM -0700, Eliot Miranda wrote:
> >  
> > On Mon, May 25, 2009 at 7:40 AM, David T. Lewis <[hidden email]> wrote:
> >
> > >
> > > On Sun, May 24, 2009 at 01:44:52PM -0700, Eliot Miranda wrote:
> > > >
> > > > 3. refactor the GeniePlugin method into an outer args parser and an inner
> > > > engine method and solve the problem.  Since VMMaker will inline anyway
> > > this
> > > > shouldn't make performance worse.  I haven't done this because I'm not
> > > the
> > > > author and it is quite a complex method.
> > >
> > > This sounds like the right thing to do, regardless of what may change
> > > in the compiler(s).
> >
> >
> > +1
> >
>
> Well unfortunately the refactoring turns out to be easier said than done.
> At least it was easier for me to say it than it was for me to do it.
>
> I made a naive attempt to split the primitive into smaller methods, but this
> just ended up trading off too many things on the stack for too many arguments
> in an argument list.
>
> The method temp variables could be changed to be instance variables. This
> would probably work fine for any current VMs (not sure about Hydra), but
> it does not pass the smell test IMO.
>
> I suspect that Nathaniel Schaerli knew what he was doing when he wrote
> this primitive, and I most certainly do not, so I don't expect that I will
> make much further progress on this front.
>
> Dave
Reply | Threaded
Open this post in threaded view
|

Re: Fwd: [Pharo-project] Could not load VMMaker in 10315

Juan Vuletich-4
 
David T. Lewis wrote:

>  
> VMMaker (from SqueakSource) can now be loaded into closure-enabled images.
>
> I was able to remove a couple of temp variables from the GeniePlugin
> primitive without affecting the logic. This was discussed on several
> lists back in May.
>
> The fix is in VMMaker-dtl.135 on SqueakSource, and is documented in
> Mantis http://bugs.squeak.org/view.php?id=7384.
>
> Dave
>  

Great Dave!

Cheers,
Juan Vuletich
Reply | Threaded
Open this post in threaded view
|

Re: Fwd: [Pharo-project] Could not load VMMaker in 10315

Eliot Miranda-2
In reply to this post by David T. Lewis
 


On Tue, Aug 18, 2009 at 5:05 AM, David T. Lewis <[hidden email]> wrote:

VMMaker (from SqueakSource) can now be loaded into closure-enabled images.

I was able to remove a couple of temp variables from the GeniePlugin
primitive without affecting the logic. This was discussed on several
lists back in May.

The fix is in VMMaker-dtl.135 on SqueakSource, and is documented in
Mantis http://bugs.squeak.org/view.php?id=7384.

Thanks, Dave, that's great!
 


Dave

On Mon, May 25, 2009 at 10:37:15PM -0400, David T. Lewis wrote:
>
> On Mon, May 25, 2009 at 11:06:10AM -0700, Eliot Miranda wrote:
> >
> > On Mon, May 25, 2009 at 7:40 AM, David T. Lewis <[hidden email]> wrote:
> >
> > >
> > > On Sun, May 24, 2009 at 01:44:52PM -0700, Eliot Miranda wrote:
> > > >
> > > > 3. refactor the GeniePlugin method into an outer args parser and an inner
> > > > engine method and solve the problem.  Since VMMaker will inline anyway
> > > this
> > > > shouldn't make performance worse.  I haven't done this because I'm not
> > > the
> > > > author and it is quite a complex method.
> > >
> > > This sounds like the right thing to do, regardless of what may change
> > > in the compiler(s).
> >
> >
> > +1
> >
>
> Well unfortunately the refactoring turns out to be easier said than done.
> At least it was easier for me to say it than it was for me to do it.
>
> I made a naive attempt to split the primitive into smaller methods, but this
> just ended up trading off too many things on the stack for too many arguments
> in an argument list.
>
> The method temp variables could be changed to be instance variables. This
> would probably work fine for any current VMs (not sure about Hydra), but
> it does not pass the smell test IMO.
>
> I suspect that Nathaniel Schaerli knew what he was doing when he wrote
> this primitive, and I most certainly do not, so I don't expect that I will
> make much further progress on this front.
>
> Dave