#recreateSpecialObjectsArray woes

Hello,

i attempted to load Alien-Core package into trunk image, and it failed.

One of the problem is override of recreateSpecialObjectsArray method..

I wonder, maybe we should design a nicer API for extending/reserving
slots in special objects array,
so it won't require overriding this method by every package which
needs to extend special objects, like FFI or Alien.

Another aspect of replacing this method is , that it makes problematic
to unload such package, in case one wants to unload it.

So, my proposal is to add a simple API to SmalltalkImage class, and in
order to install additional special objects one should do like:

MyClass class>>initialize

Smalltalk registerSpecialObject: myObject at: 55.
Smalltalk registerSpecialObject: myObject at: 56.
...

or:

Smalltalk registerSpecialObjects: #( x y z) startingAt: 56.

upon package unload, a reverse scheme should be supported:


MyClass class>>unload

Smalltalk unregisterSpecialObjectAt: 55.
Smalltalk unregisterSpecialObjectAt: 56.
...

or:

Smalltalk unregisterSpecialObjects: 3 startingAt: 56.


what you think?

--
Best regards,
Igor Stasenko AKA sig.

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

On Apr 12, 2010, at 4:17 11PM, Igor Stasenko wrote:

+1 for the overall idea.
Implementation-wise, I'd also vote for raising errors in registerSpecialObject:at: if something other than myObject is already registered, changing to unregisterSpecialObject: at: in order to not unintentionally unregistering something else, and disallow unregisterSpecialObjects:startingAt: ;)

Cheers,
Henry


_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

Good idea! I had a similar idea when initially worked on Alien loader.

I think that what is needed also is remove the dependency with the array, i mean explicitly calling the indexes make the image crash if you didn't had the  assumed special object array size.


For example:
        Smalltalk registerSpecialObject: myObject with: aName

        And later you reference that special object by aName instead of an index to the array.


So it would be easier to load/unload the packages that extend the special objects array, because there would be no explicit dependency to indexes.


Fernando

On Apr 12, 2010, at 4:17 PM, Igor Stasenko wrote:


_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

On 12 April 2010 17:43, Fernando olivero <[hidden email]> wrote: Yeah, that would be nice.
But this will require VM changes.. and therefore is doomed :)
Also, this could impact the VM speed significantly, because its using
special object quite frequently.
Imagine replacing a simple index-based lookup with name(string) based.



--
Best regards,
Igor Stasenko AKA sig.

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

On Apr 12, 2010, at 5:16 PM, Igor Stasenko wrote:



Hi Igor, i wasn't speaking of an on-image API change.

For example if you introduced the API you mentioned we could replace

Utilities>> initializeClosures
        "Eliminate the prototype BlockContext from the specialObjectsArray.  The VM doesn't use it. This paves the way for removing BlockCOntext altogether and merging ContextPart and MethodContext into e.g. Context."
        (Smalltalk specialObjectsArray at: 38) class == BlockContext
                ifTrue:[Smalltalk specialObjectsArray at: 38 put: nil].

With

Utilities>> initializeClosures
        ......
        (Smalltalk hasSpecialObjectNamed: #BlockContext)
                ifTrue:[ Smalltalk unregisterSpecialObjectNamed: #BlockContext ].
        ....

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

+ 1 for that.

Stef

On Apr 12, 2010, at 4:42 PM, Henrik Johansen wrote:


_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

On 12 April 2010 18:30, Fernando olivero <[hidden email]> wrote:
Ah you mean associating indexes with corresponding names, so code can
use a nice-looking names
instead of bare indexes?
But then who is responsible for providing these associations?
Somewhere you have to put things like:
BlockContextIndex :=  38.

so, then later you can do nice:
Smalltalk unregisterSpecialObjectNamed: #BlockContextIndex

The problem is, that we don't know these names beforehead.
So, should a newly loaded package also define a new index names then?

Like

Smalltalk registerSpecialObjectIndex: 55  named: #MyName.

and then to register an object, it should do:

Smalltalk specialObjectAt: #MyName put: MySpecialObject.

?


> _______________________________________________
> Pharo-project mailing list
> [hidden email]
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>



--
Best regards,
Igor Stasenko AKA sig.

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

On 12 April 2010 19:32, Andreas Raab <[hidden email]> wrote:
> On 4/12/2010 7:17 AM, Igor Stasenko wrote:
>>
>> what you think?
>
> I think "no, no, no". Two reasons: 1) the extension of the splObjs implies a
> hardcoded contract with the VM; having it externally settable makes it look
> as if one could just stick in stuff for fun.

By 'For fun', you mean writing a plugin, which needs own a special
objects to work?
I wanted to add a special objects couple of times in plugins, but then i had to
step back , knowing that it will require overriding the #recreateSpecialObjects
method, which is tedious and error prone, because you never know if people
who may use your plugin could also use FFI, Alien or something else.

By keeping things intact, we're having a bottleneck which doesn't
allows developers
to use a special objects array in their plugins for own purposes (i
guess you're not
assuming that FFI and Alien are the only exceptional plugins which
having an ultimate right to
touch a special objects array).

> 2) The recreation of the
> splObjs implies specific knowledge about which values should be added and
> which values should be removed.
>
Sure, each part of a system knows wery well which objects should be
there and at what place.
But there is no need to recreate an array (unless it needs to grow) ,
and it could be done automatically,
by copying old objects into a new, bigger array, while revoking the
#recreateSpecialObjects from system completely.
For replacing an object(s) in array there is a simple as #at:put:
Ensuring an array size doesn't means automatically that it should be
rebuilt from scratch, in same way as OrderedCollection
doesn't forcing you to add all items from scratch into it, once it
needs to grow.

> In any case, there are only two packages that have ever made mods to the
> splObjs array (FFI and Alien) for *very* specialized reasons. We should just
> add these definitions manually and remove the patch from the Alien package.
>
I think you are mixing the cause and consequence here. As to me, the
reason, why there's only two
plugins who extending a special objects array is because its very
tedious and error prone.
Mostly because of conflicting overrides by different packages!

Anyways, if you don't like playing with special objects array. Please,
can we think of a scheme which at least
allows us to avoid overrides of a very same method by different packages?

> Cheers,
>  - Andreas
>


--
Best regards,
Igor Stasenko AKA sig.

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

On 4/12/2010 10:43 AM, Igor Stasenko wrote:
You don't need special objects. Their use in FFI and Alien is a
historical accident. What you'd do today is register the classes
directly with your plugin instead of sticking them into splObjs.

> I wanted to add a special objects couple of times in plugins, but then i had to
> step back , knowing that it will require overriding the #recreateSpecialObjects
> method, which is tedious and error prone, because you never know if people
> who may use your plugin could also use FFI, Alien or something else.

Precisely. All of these are good reasons why you shouldn't modify
splObjs to begin with. What you do instead is to provide a primitive in
your plugin that accepts the classes / objects you intend to use. This
wouldn't work in the past because you had no way of handling GC correctly.

Today, you can give your plugin an array of your own special objects
that is used by your plugin, registered with the VM as a GC root and
there's no need to modify splObjs.

Cheers,
   - Andreas


_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

On 12 April 2010 20:49, Andreas Raab <[hidden email]> wrote: Looks like i missed something. HOW?
Suppose a primitive needs to answer an instance of a special class, known by it.
How it can instantiate it, if it having no any clues, where its
invoked from, and it can't rely
on a receiver, nor any of method's arguments?

> Cheers,
>  - Andreas
>
>>



--
Best regards,
Igor Stasenko AKA sig.

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

On 4/12/2010 10:57 AM, Igor Stasenko wrote:
> On 12 April 2010 20:49, Andreas Raab<[hidden email]>  wrote:
>> Today, you can give your plugin an array of your own special objects that is
>> used by your plugin, registered with the VM as a GC root and there's no need
>> to modify splObjs.
>>
> Looks like i missed something. HOW?
> Suppose a primitive needs to answer an instance of a special class, known by it.
> How it can instantiate it, if it having no any clues, where its
> invoked from, and it can't rely on a receiver, nor any of method's arguments?

You make it work just like splObjs, i.e.,

FooPlugin>>primitiveSetSplObjs
        "Sets the special objects for my plugin"
        splObjs := interpreterProxy stackObjectValue: 0.
        "... should have sanity checks ..."
        interpreterProxy addGCRoot: (self cCode: '&splObjs').

FooPlugin>>instantiateNewBar
        "Instantiates a new Bar (at index 2 in the splObjs)"

        "... should have sanity checks ..."
        foo := interpreterProxy instantiateClass: (interpreterProxy
fetchPointer: 2 ofObject: splObjs) indexableSize: 0.
        interpreterProxy pop: 1 thenPush: foo.

then you use it like here:

Foo class>>startUp: resuming
        "Install my special objects on system startup"
        resuming ifTrue:[
                self primitiveSetSplObjs: {Foo. Bar. DukeNukem. 42}.
        ].

Foo class>>newBar
        ^self primitiveNewBar

The "trick" is that your plugin can register whatever it wants for GC
tracing (in fact you should probably set the variable to nilObj and call
addGCRoot from initialiseModule exactly once and release it via
shutdownModule). This removes the need for abusing splObjs to stick in
stuff that you'd like to preserve between prim calls.

Cheers,
   - Andreas

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

On 12 April 2010 21:08, Andreas Raab <[hidden email]> wrote:
Ah, that's cool.. Except that most recent SVN repository
sqVirtualMachine.h (revision 1955) having no addGCroot: defined in
InterpreterProxy struct.
I am using wrong SVN repository?
http://squeakvm.org/svn/squeak/trunk/

After some research i found another file in plugins/IA32ABI/, which
includes new proxy functions,  but its not used by builder.

> Cheers,
>  - Andreas
>
>



--
Best regards,
Igor Stasenko AKA sig.

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

On 4/12/2010 11:46 AM, Igor Stasenko wrote:
> Ah, that's cool.. Except that most recent SVN repository
> sqVirtualMachine.h (revision 1955) having no addGCroot: defined in
> InterpreterProxy struct.
> I am using wrong SVN repository?
> http://squeakvm.org/svn/squeak/trunk/

Err, weird. The code is in VMMaker but it seems as if SVN hasn't been
updated. It should include this:

#if VM_PROXY_MINOR > 7
   /* New methods for proxy version 1.8 */

   /* callbackEnter: Re-enter the interpreter loop for a callback.
      Arguments:
        callbackID: Pointer to a location receiving the callback ID
                    used in callbackLeave
      Returns: True if successful, false otherwise */
   sqInt (*callbackEnter)(sqInt *callbackID);

   /* callbackLeave: Leave the interpreter from a previous callback
      Arguments:
        callbackID: The ID of the callback received from callbackEnter()
      Returns: True if succcessful, false otherwise. */
   sqInt (*callbackLeave)(sqInt  callbackID);

   /* addGCRoot: Add a variable location to the garbage collector.
      The contents of the variable location will be updated accordingly.
      Arguments:
        varLoc: Pointer to the variable location
      Returns: True if successful, false otherwise. */
   sqInt (*addGCRoot)(sqInt *varLoc);

   /* removeGCRoot: Remove a variable location from the garbage collector.
      Arguments:
        varLoc: Pointer to the variable location
      Returns: True if successful, false otherwise.
   */
   sqInt (*removeGCRoot)(sqInt *varLoc);
#endif

Cheers,
   - Andreas

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

On 12 April 2010 21:52, Andreas Raab <[hidden email]> wrote:
Yes. I remember seeing that code before.
This is twice strange that its not in SVN repo.

[snip]

>
> Cheers,
>  - Andreas
>
>



--
Best regards,
Igor Stasenko AKA sig.

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

It's in the Alien directory. somehow it never got promoted to the cross platform directory.
So check the sqVirtualMachine.h   You  may need to adjust it.

On 2010-04-12, at 12:01 PM, Igor Stasenko wrote:
--
===========================================================================
John M. McIntosh <[hidden email]>   Twitter:  squeaker68882
Corporate Smalltalk Consulting Ltd.  http://www.smalltalkconsulting.com
===========================================================================





_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

Okay, then i propose to:

1. fix the interpreterProxy spec, so all VMs will include addRoot() & friends
2. create a single and _final_ version of #recreateSpecialObjectsArray
3. remove overrides of this method from both FFI and Alien packages.
4. Optionally, rewrite plugins to use own special objects at initialization, so
they no longer need extending specialObjectsArray, and hence use of
#recreateSpecialObjectsArray

--
Best regards,
Igor Stasenko AKA sig.

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

On 4/12/2010 12:49 PM, Igor Stasenko wrote:
> Okay, then i propose to:
>
> 1. fix the interpreterProxy spec, so all VMs will include addRoot()&  friends
> 2. create a single and _final_ version of #recreateSpecialObjectsArray
> 3. remove overrides of this method from both FFI and Alien packages.
> 4. Optionally, rewrite plugins to use own special objects at initialization, so
> they no longer need extending specialObjectsArray, and hence use of
> #recreateSpecialObjectsArray

+1.

Cheers,
   - Andreas

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

On Apr 12, 2010, at 9:49 PM, Igor Stasenko wrote:

> Okay, then i propose to:
>
> 1. fix the interpreterProxy spec, so all VMs will include addRoot() & friends
> 2. create a single and _final_ version of #recreateSpecialObjectsArray
> 3. remove overrides of this method from both FFI and Alien packages.
> 4. Optionally, rewrite plugins to use own special objects at initialization, so
> they no longer need extending specialObjectsArray, and hence use of
> #recreateSpecialObjectsArray

5. Add a **large comment** stating how it should be done. For example using the example
andreas used.


Stef

>
> --
> Best regards,
> Igor Stasenko AKA sig.
>
> _______________________________________________
> Pharo-project mailing list
> [hidden email]
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project


_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project