Keeping oops across primitives

Hi Folks -

After having dodged the problem multiple times, I now really need a
solution that allows one to keep references to Squeak objects from
external (plugin) code across primitive calls.

What I've been thinking here is that the simplest possible solution
would consist of an interface that merely allows a plugin to add an
array of OOPs to the interpreter roots. This would mean that such an
array is traversed and remapped just like the other interpreter oops but
that the plugin can now refer to those oops throughout its lifetime and
keep strong references to any objects it needs (such as well-known
classes etc).

Interface:
The plan would be to extend the interpreter proxy by the following function:

sqInt addToVmRoots(sqInt *newArray, sqInt newCount, sqInt *oldArray);
   newArray: Pointer to an array of oops traversed by the collector
   newCount: Size of newArray
   oldArray: The oop array we're trying to replace in this call

Following the above we have the following operations:
   - add to the roots:
     vm->addToVmRoots(newArray, newCount, NULL); /* replace nothing */

   - resize/change an existing array:
     vm->addToVmRoots(newArray, newCount, oldArray);

   - delete/remove an existing array:
     vm->addToVmRoots(NULL, 0, oldArray); /* replace with NULL */

Tradeoffs:
Pro: Really simple to implement in the VM. Basically everything we need
is a list of pointers to traceable arrays which get hooked into
mapAndTraceInterpreterOops.
Pro: Really simple to maintain from the plugin side. The plugin can
decide how many entries it needs (to keep overhead small) and if it
wants to preserve an oop it just sticks it into that array and
everything is fine.
Pro: The plugin keeps full control over the life-time of the object. The
image can't pull out the rugs under the plugin if the plugin thinks that
this oop may still be needed.
Con: Adding to the roots increases the overhead of the collector but
that's something I'm willing to live with since if that ever gets an
issue another indirection (through a Squeak array for example) can help
to mitigate this problem.

Question: Does anyone see a serious problem with the above proposal? I'm
about to implement this right away so if you see an issue with it,
please let me know.

Cheers,
   - Andreas

Cees De Groot wrote:
> I'm hardly a VM hacker, but:
>
> On 6/6/06, Andreas Raab <[hidden email]> wrote:
>> sqInt addToVmRoots(sqInt *newArray, sqInt newCount, sqInt *oldArray);
>
> why not oldCount as an argument? Or zero-terminated arrays for that matter?

oldCount is irrelevant since the VM won't copy the contents of newArray;
it will merely keep a reference to it (the array really is under control
of the plugin). Therefore all we need is the reference to oldArray (this
also goes for reallocating via realloc() - just remember the old pointer
before you do that and pass it into the above to update the oop array).

Why not NULL terminated? I'm planning on using NULL as the "undefined
entry" in the oop array. Since this code is managed by some other C code
that seems preferrable to nil (although the plugin could store nil just
as well). But it probably will be valuable for the C code to be able to
say "clear this entry" via storing NULL and having the VM skip this is
fairly simple.

Cheers,
   - Andreas

Andreas Raab wrote:

> Why not NULL terminated? I'm planning on using NULL as the "undefined
> entry" in the oop array. Since this code is managed by some other C
> code that seems preferrable to nil (although the plugin could store
> nil just as well). But it probably will be valuable for the C code to
> be able to say "clear this entry" via storing NULL and having the VM
> skip this is fairly simple.

Perhaps it would be easier for the VM if you'd use an immediate value
for the undefined entry so that all entries were valid oops?
I don't know how exactly the code for this would be merged into the
existing GC stuff, but to me at least it feels like it would be easiest
if there were as few as possible special cases, and the GC already
distinguishes between immediate values and object pointers.

Cheers,
Hans-Martin

Hans-Martin Mosner wrote:
Like I was saying, the plugin can do that (just store nilObj in the
slot). I'm just not going to require it because the NULL test is going
to be faster (and more easily understood by the C programmer).

> I don't know how exactly the code for this would be merged into the
> existing GC stuff, but to me at least it feels like it would be easiest
> if there were as few as possible special cases, and the GC already
> distinguishes between immediate values and object pointers.

Actually, that's not relevant here - we're talking about remapping
storage locations (variables) that are managed by the plugin. So code
would do the logical equivalent of:

   /* in markAndTraceInterpreterOops */
   if(pluginArray[i]) markAndTrace(pluginArray[i]);

   /* in mapInterpreterOops */
   if(pluginArray[i]) pluginArray[i] = remap(pluginArray[i]);

Whether the contents of pluginArray[i] is an immediate or not isn't
really a concern for that code.

Cheers,
   - Andreas