UFFI and autoRelease

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

UFFI and autoRelease

SergeStinckwich

I'm using autoRelease on some FFIOpaqueObject instances.

I need to test some behavior when I delete explicitly one of these objects. ​How to remove these objects from the finalization list, in order they are not freed two times.

Something like ignoreFinalization ?

--
Serge Stinckwich
UMI UMMISCO 209 (SU/IRD/UY1)
"Programs must be written for people to read, and only incidentally for machines to execute."
http://www.doesnotunderstand.org/
Reply | Threaded
Open this post in threaded view
|

Re: UFFI and autoRelease

EstebanLM
hi,

there is nothing like that and I do not recommend messing with the registry in general.
but… you can always extend the classes for your use, is around FFIExternalResourceManager.

cheers,
Esteban


On 24 Apr 2018, at 14:48, Serge Stinckwich <[hidden email]> wrote:


I'm using autoRelease on some FFIOpaqueObject instances.

I need to test some behavior when I delete explicitly one of these objects. ​How to remove these objects from the finalization list, in order they are not freed two times.

Something like ignoreFinalization ?

--
Serge Stinckwich
UMI UMMISCO 209 (SU/IRD/UY1)
"Programs must be written for people to read, and only incidentally for machines to execute."
http://www.doesnotunderstand.org/

Reply | Threaded
Open this post in threaded view
|

Re: UFFI and autoRelease

SergeStinckwich
I don't understand the default

finalizeResourceData: handle
    handle isNull ifTrue: [ ^ self ].
    handle free.
    handle beNull


when handle is an ExternaData, free is defined as in ExternalStructure class as:

free
    "Free the handle pointed to by the receiver"
    (handle ~~ nil and:[handle isExternalAddress]) ifTrue:[handle free].
    handle := nil.


but beNull is not defined on nil.

I can define my own finalizeResourceData: but the default one, does not work apparently.



On Tue, Apr 24, 2018 at 2:06 PM, Esteban Lorenzano <[hidden email]> wrote:
hi,

there is nothing like that and I do not recommend messing with the registry in general.
but… you can always extend the classes for your use, is around FFIExternalResourceManager.

cheers,
Esteban


On 24 Apr 2018, at 14:48, Serge Stinckwich <[hidden email]> wrote:


I'm using autoRelease on some FFIOpaqueObject instances.

I need to test some behavior when I delete explicitly one of these objects. ​How to remove these objects from the finalization list, in order they are not freed two times.

Something like ignoreFinalization ?

--
Serge Stinckwich
UMI UMMISCO 209 (SU/IRD/UY1)
"Programs must be written for people to read, and only incidentally for machines to execute."
http://www.doesnotunderstand.org/




--
Serge Stinckwich
UMI UMMISCO 209 (SU/IRD/UY1)
"Programs must be written for people to read, and only incidentally for machines to execute."
http://www.doesnotunderstand.org/
Reply | Threaded
Open this post in threaded view
|

Re: UFFI and autoRelease

Ben Coman


On 25 April 2018 at 23:08, Serge Stinckwich <[hidden email]> wrote:
I don't understand the default

finalizeResourceData: handle
    handle isNull ifTrue: [ ^ self ].
    handle free.
    handle beNull


when handle is an ExternaData, free is defined as in ExternalStructure class as:

free
    "Free the handle pointed to by the receiver"
    (handle ~~ nil and:[handle isExternalAddress]) ifTrue:[handle free].
    handle := nil.

Theorectical side thought.   I wonder how threadsafe that code for #free is?
Sending two messages between the ~~nil test and nil-assignment 
seems to leave the door open for another thread to erroneously pass the ~~nil test.

IIUC, with an inlined #ifNotNil, the following might be safer (?)

    free
        "Save handle so its nil assignment is inlined with its #ifNotNil: test, eliminating possible thread change during message sends"
        |savedhandle|
        handle ifNotNil: [
            savedHandle := handle.
            handle := nil.
            savedHandle isExternalAddress ifTrue: [ savedHandle  free].
            ].

cheers -ben
 


but beNull is not defined on nil.

I can define my own finalizeResourceData: but the default one, does not work apparently.



On Tue, Apr 24, 2018 at 2:06 PM, Esteban Lorenzano <[hidden email]> wrote:
hi,

there is nothing like that and I do not recommend messing with the registry in general.
but… you can always extend the classes for your use, is around FFIExternalResourceManager.

cheers,
Esteban


On 24 Apr 2018, at 14:48, Serge Stinckwich <[hidden email]> wrote:


I'm using autoRelease on some FFIOpaqueObject instances.

I need to test some behavior when I delete explicitly one of these objects. ​How to remove these objects from the finalization list, in order they are not freed two times.

Something like ignoreFinalization ?

--
Serge Stinckwich
UMI UMMISCO 209 (SU/IRD/UY1)
"Programs must be written for people to read, and only incidentally for machines to execute."
http://www.doesnotunderstand.org/




--
Serge Stinckwich
UMI UMMISCO 209 (SU/IRD/UY1)
"Programs must be written for people to read, and only incidentally for machines to execute."
http://www.doesnotunderstand.org/

Reply | Threaded
Open this post in threaded view
|

Re: UFFI and autoRelease

SergeStinckwich


On Wed, Apr 25, 2018 at 11:07 PM, Ben Coman <[hidden email]> wrote:


On 25 April 2018 at 23:08, Serge Stinckwich <[hidden email]> wrote:
I don't understand the default

finalizeResourceData: handle
    handle isNull ifTrue: [ ^ self ].
    handle free.
    handle beNull


when handle is an ExternaData, free is defined as in ExternalStructure class as:

free
    "Free the handle pointed to by the receiver"
    (handle ~~ nil and:[handle isExternalAddress]) ifTrue:[handle free].
    handle := nil.

Theorectical side thought.   I wonder how threadsafe that code for #free is?
Sending two messages between the ~~nil test and nil-assignment 
seems to leave the door open for another thread to erroneously pass the ~~nil test.

IIUC, with an inlined #ifNotNil, the following might be safer (?)

    free
        "Save handle so its nil assignment is inlined with its #ifNotNil: test, eliminating possible thread change during message sends"
        |savedhandle|
        handle ifNotNil: [
            savedHandle := handle.
            handle := nil.
            savedHandle isExternalAddress ifTrue: [ savedHandle  free].
            ].



​You don't reply to my original question, Ben :-)


I have another concern with UFFI !

When I call ffiCall: and the call returns an Opaque Object, an instance of this Opaque object is created, but apparently,
basicNew is called instead of new on the class. What is the ​
 
​intention behind ?​

​Thank you.​

--
Serge Stinckwich
UMI UMMISCO 209 (SU/IRD/UY1)
"Programs must be written for people to read, and only incidentally for machines to execute."
http://www.doesnotunderstand.org/
Reply | Threaded
Open this post in threaded view
|

Re: UFFI and autoRelease

EstebanLM


On 26 Apr 2018, at 12:09, Serge Stinckwich <[hidden email]> wrote:



On Wed, Apr 25, 2018 at 11:07 PM, Ben Coman <[hidden email]> wrote:


On 25 April 2018 at 23:08, Serge Stinckwich <[hidden email]> wrote:
I don't understand the default 

finalizeResourceData: handle
    handle isNull ifTrue: [ ^ self ].
    handle free.
    handle beNull


when handle is an ExternaData, free is defined as in ExternalStructure class as: 

free
    "Free the handle pointed to by the receiver"
    (handle ~~ nil and:[handle isExternalAddress]) ifTrue:[handle free].
    handle := nil. 

Theorectical side thought.   I wonder how threadsafe that code for #free is?
Sending two messages between the ~~nil test and nil-assignment 
seems to leave the door open for another thread to erroneously pass the ~~nil test.

IIUC, with an inlined #ifNotNil, the following might be safer (?)

    free
        "Save handle so its nil assignment is inlined with its #ifNotNil: test, eliminating possible thread change during message sends"
        |savedhandle|
        handle ifNotNil: [
            savedHandle := handle.
            handle := nil.
            savedHandle isExternalAddress ifTrue: [ savedHandle  free].
            ].



​You don't reply to my original question, Ben :-)


I have another concern with UFFI !

When I call ffiCall: and the call returns an Opaque Object, an instance of this Opaque object is created, but apparently,
basicNew is called instead of new on the class. What is the ​
 
​intention behind ?​

basicNew ensures the object is created “virgin”, since its contents will depend on what comes from outside and not in Pharo. 
you can then call initialize by yourself if needed, as it happens for example in AthensCairoSurface>>extent:format: 

Esteban


​Thank you.​

-- 
Serge Stinckwich
UMI UMMISCO 209 (SU/IRD/UY1)
"Programs must be written for people to read, and only incidentally for machines to execute."
http://www.doesnotunderstand.org/

Reply | Threaded
Open this post in threaded view
|

Re: UFFI and autoRelease

Ben Coman
In reply to this post by SergeStinckwich


On 26 April 2018 at 18:09, Serge Stinckwich <[hidden email]> wrote:
>
> You don't reply to my original question, Ben :-)

I didn't know the answer :P   
and I wasn't near an Image where I could poke around and speculate.  
Having a poke around now, but my thoughts are speculative - the few times I've 
used FFI I've muddled through rather than having a good grasp of it.


>> On 24 Apr 2018, at 14:48, Serge Stinckwich <[hidden email]> wrote:
>>
>>
>> I'm using autoRelease on some FFIOpaqueObject instances.
>>
>> I need to test some behavior when I delete explicitly one of these objects. How to remove these objects from the finalization list, in order they are not freed two times.
>>
>> Something like ignoreFinalization ?

On 25 April 2018 at 23:08, Serge Stinckwich <[hidden email]> wrote:
>
> I don't understand the default
>
> finalizeResourceData: handle
>     handle isNull ifTrue: [ ^ self ].
>     handle free.
>     handle beNull
>
>
> when handle is an ExternaData, free is defined as in ExternalStructure class as:
>
> free
>     "Free the handle pointed to by the receiver"
>     (handle ~~ nil and:[handle isExternalAddress]) ifTrue:[handle free].
>     handle := nil.
>
>
> but beNull is not defined on nil.


I presume your code is getting a DNU. Do you have a code extract to produce it?
Looking only at the static code, my I can't get by the impression that
there are two separate /handle/ variables in the above code.
i.e. The /handle/ of FFIExternalReference-class>>finalizeResourceData:  
can't be set to nil by the ExternalStructure>>free.

So, so far I can't provide a sensical answer.
But I've had trouble debugging in the past keeping my /handle/s straight in different contexts 
and found it useful to temporarily refactor instance variables names to distriguish them.
 

btw, I notice /handle/ of FFIExternalReference is defined in ExternalObject, whose class comment says...
    Instance variables:
        handle <ByteArray | ExternalAddress>

ExternalAddress subclasses ByteArray
and there is ExternalAddress>>beNull 
but no ByteArray>>beNull,
So it seems   FFIExternalReference  further restricts /handle/ to be *only* ExternalAddress,
which makes some sense (Reference~Addresss).

Those comments don't indicate ExternalData is valid in /handle/ 
also indicated by ExternalData does not define #beNull.
So I've got as far as wondering if either:
* the class comment need updating, or
* how ExternalData gets into /handle/ and is that a valid path.

Sorry, not much help so far.

cheers -ben

>
> I can define my own finalizeResourceData: but the default one, does not work apparently
>
> On Tue, Apr 24, 2018 at 2:06 PM, Esteban Lorenzano <[hidden email]> wrote:
>>
>> hi,
>>
>> there is nothing like that and I do not recommend messing with the registry in general.
>> but… you can always extend the classes for your use, is around FFIExternalResourceManager.
>>
>> cheers,
>> Esteban


P.S.
Is your Image crashing from a double-free?   
I not been able to reproduce an observation from a while ago, 
but once I came across the object with an external memory reference that had been copied,
so when the first object was #free'd, releasing external memory and setting /handle/ to nil to guard against a double-release,
the copied-object's /handle/ was not nil, so when it was #free'd it also released the same external memory.

Which is why I wonder that object holding references to external memory should behave like symbols
where the default copy behaviour is to return itself.
i.e. References to external malloc'd memory should be treated as *unique*.  
Taking a #copy of the reference doesn't produce extra external memory.