FFI ExternalStructure - proper memory cleanup

Hello,

I have had a casual interest in squeak for a while, but just started
doing more serious development in it recently (lots of fun!).  I am
trying to bridge some functionality from the gnu scientific library
(GSL) so I can do numerical stuff in squeak.

I have successfully bridged the main functionality I need, but after
some research am still a bit confused on the proper way to deal with
memory allocated outside the VM:

1. I have ExternalStructure subclasses, e.g. GSLMatrix, that
correspond to C structures in the GSL library.  (see
http://www.gnu.org/software/gsl/manual/html_node/Matrices.html#Matrices
for the structure definition.)

2. I allocate these structures using FFI with the GSL library, e.g. via

gsl_matrix * gsl_matrix_alloc (size_t n1, size_t n2)

3. I can deallocate the external memory with an FFI call:

void gsl_matrix_free (gsl_matrix * m)

My questions is WHEN/WHERE I should make this call to make sure
external memory is cleaned up properly.  In the FFI examples and
searching the website, I have seen this done in instance methods
"free", "dispose", "destroy", as well as the "post-mortem"
finalization described here: http://wiki.squeak.org/squeak/664.

Can anyone describe or point me to the "best practice" I should use?

Similarly, I assume that if I created instances that point to external
structures (e.g. created instances in my Workspace), and shutdown
squeak, when I restart the VM, I have some instances with handles that
point to invalid external addresses.  I am OK to just avoid using
these objects, but will the fact that they still have these handles
cause problems (e.g. when garbage collection occurs)?  If so, what is
the best practice here - to reconstitute the external portion of the
objects when the image is loaded?

Thanks for any help, apologies if this is more appropriate for the
beginners list.

cheers,

Jim M

Am Jan 20, 2007 um 23:53  schrieb Jim McLoughlin:

I think #destroy is the selector most commonly used for that. You  
should send it as soon as you're done using the object.

As a safety guard, you could use finalization in addition to that  
(sending #destroy). Or make it issue a warning so you know you forgot  
to properly release something.

> Similarly, I assume that if I created instances that point to external
> structures (e.g. created instances in my Workspace), and shutdown
> squeak, when I restart the VM, I have some instances with handles that
> point to invalid external addresses.  I am OK to just avoid using
> these objects, but will the fact that they still have these handles
> cause problems (e.g. when garbage collection occurs)?  If so, what is
> the best practice here - to reconstitute the external portion of the
> objects when the image is loaded?

On system startup, I'd nil out all those dangling references in all  
instances. And allocate them lazily when they are actually used.

- Bert -

On Jan 20, 2007, at 2:53 PM, Jim McLoughlin wrote:

> Thanks for any help, apologies if this is more appropriate for the
> beginners list.
>
> cheers,
>
> Jim M
>

Doing this correctly is a complex question

If you look at MPEGFile you'll see a pattern of using weak object  
that I used a number of years back.

The class has an class variable Registry
Class side methods
registry
register:
unregister:
are used to register and unregister: instances

when an instance is created or destroyed
register
and unregister instance side methods are used to place or remove the  
object in the Registry class variable.

On register, a shallowcopy of the instance is created which is held  
onto by the Registery weakly. When the
original instance is garbage collected, the weak entry will be  
finalized.

An important issue here is that the instance variables in the copy  
did not create a circular reference to
the original, which will prevent the copy from being Gced. see  
StandardFileStream>actAsExecutor
Usually one does not have an issue, you'll know because your copy  
instance won't go away if you do.

Usually then when you explicity destroy your instance, you would  
invoke unregister to remove the instance
from the weak array registery.

On finalization the copy instance is sent finalize where you would do  
any cleanup required like the destroy of external
resources, and unregisterExternalObject:


As for the external address. You'll also note in the MPEGFile we do

fileIndex := Smalltalk registerExternalObject: fileBits.
self register.

when we create the instance

The registerExternalObject: is registering the results of a primitive  
call in a special VM table that is cleared when
the VM is restarted.  Later we use the Smalltalk externalObjects  
at:ifAbsent: to get the bits out of the VM Table using the fileIndex
value we remember . I'll note since it's slot based it could return  
bits from something else, so that is why we check against fileBits.

        self fileHandle = fileBits ifTrue: [Smalltalk  
unregisterExternalObject: fileIndex].


fileHandle
        (Smalltalk externalObjects at: fileIndex ifAbsent: [^nil]) == fileBits
                ifTrue: [^fileBits]
                ifFalse: [^nil].


You can of course as earlier noted have a class side startup method  
an nil out instance vars for all instances when the VM is started...

--
========================================================================
===
John M. McIntosh <[hidden email]>
Corporate Smalltalk Consulting Ltd.  http://www.smalltalkconsulting.com
========================================================================
===

Thanks to Bert and John for the quick responses, I think I understand
how to handle cleaning up my instances.

> You can of course as earlier noted have a class side startup method
> an nil out instance vars for all instances when the VM is started...

Anyone have a quick example on how to do this?  I tried to put it in
my GSLMatrix class>>initialize, which already exists to define the
structure fields.  I also tried declaring a class side method startUp,
but neither seemed to be invoked on image restart.

Did I need to somehow register the startUp method with the image
startup process?

Jim

Sure in the class initialization method

        Smalltalk addToStartUpList: self.
or perhaps
        Smalltalk addToStartUpList: self after: ExternalSettings.
to ensure you are added to the master startup list somewhere rationally.

I'll note in Sophie we had an interesting issue once where the  
startup logic for Rome based Fonts was startedUp via just the  
addToStartupList:
but one day we rebuilt a test image and discovered if it managed to  
add the rome logic AFTER morphic startup which would paint the  
windows we were hosed.
Therefore we had to ensure we added our rome logic BEFORE morphic .

and of course startup would be

startUp: resuming
        resuming
                ifFalse: [^ self].
        self dosomething...


Ah and resuming is true when... Exercise for the reader, one case is  
when you save the image, one case is when the image is restarted by  
restarting the virtual machine.

On Jan 20, 2007, at 7:53 PM, Jim McLoughlin wrote:

--
========================================================================
===
John M. McIntosh <[hidden email]>
Corporate Smalltalk Consulting Ltd.  http://www.smalltalkconsulting.com
========================================================================
===