Smalltalk › Squeak › Squeak VM

Move to Spur: how can I change WeakRegistry packages?

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

6 messages Options

EstebanLM

Move to Spur: how can I change WeakRegistry packages?

Hi,

I need to change the WeakRegistry, to adapt Pharo to finalisation on Spur and I wanted to “star over”, injecting into Pharo the packages Collections-Weak and System-Finalizer from Squeak.
Now, that breaks everything, of course :)

So, I wonder if there is a clean way to do this?

thanks,
Esteban

Eliot Miranda-2

Re: Move to Spur: how can I change WeakRegistry packages?

Hi Esteban,

On Wed, Sep 30, 2015 at 5:39 AM, Esteban Lorenzano <[hidden email]> wrote:

Hi,

I need to change the WeakRegistry, to adapt Pharo to finalisation on Spur and I wanted to “star over”, injecting into Pharo the packages Collections-Weak and System-Finalizer from Squeak.
Now, that breaks everything, of course :)

So, I wonder if there is a clean way to do this?

That's a great question. A few weeks ago I implemented the Ephemeron class for Squeak (I haven't committed it yet) and set about changing the finalization scheme to support both ephemerons and non-weak-registry-based weak array finalization. The latter is where the weak array that needs finalizing is added to the same VM queue that an ephemeron that needs finalizing gets added to. So instead of all WeakArrays getting finalized via the WeakRegistry mechanism every time one or more weak arrays get finalized, only the weak arrays that have actually lost a reference will get finalized. Alas, for reasons I don't understand, this broke the file system. I would use the system for some minutes and then suddenly source code viewed in the browser would be decompiled, since the source file had closed. Alas I don't have time to look at this now, but I do think very much it is the right way to go. Right now I'm busy with a long list of dependent changes that I must get fixed to make progress. So for now I can't help other than to e.g. mail you, or post, my prototype finalization changes and have you look at those and see if you can get them working in Pharo. I;m also interested in getting anyone else who would like to help involved. I am so overloaded right now it's not funny [right, ed :-) "Get some rest. If you don't have your health, you don't have anything". Count Rugen]

thanks,
Esteban

_,,,^..^,,,_

best, Eliot

EstebanLM

Re: Move to Spur: how can I change WeakRegistry packages?

Hi, Eliot

So, I was wanting to adopt Squeak version (since our version are working with some Igor changes that does not apply anymore…)

This started because in my FFI-NB stuff, I realised finalisers are not called at all, and I assumed it was because of the different versions.

Right now, I’m struggling to inject Squeak sources into Pharo without break everything in the middle :(

Since I cannot continue without this working, and I cannot finish migration of Pharo to spur without a FFI-NB implementation, I’m willing to spend all time I can on it.

So… yes please, send me your prototype and I will try to understand what is missing :)

Esteban

On 30 Sep 2015, at 19:15, Eliot Miranda <[hidden email]> wrote:

Hi Esteban,

On Wed, Sep 30, 2015 at 5:39 AM, Esteban Lorenzano <[hidden email]> wrote:

Hi,

I need to change the WeakRegistry, to adapt Pharo to finalisation on Spur and I wanted to “star over”, injecting into Pharo the packages Collections-Weak and System-Finalizer from Squeak.
Now, that breaks everything, of course :)

So, I wonder if there is a clean way to do this?

That's a great question. A few weeks ago I implemented the Ephemeron class for Squeak (I haven't committed it yet) and set about changing the finalization scheme to support both ephemerons and non-weak-registry-based weak array finalization. The latter is where the weak array that needs finalizing is added to the same VM queue that an ephemeron that needs finalizing gets added to. So instead of all WeakArrays getting finalized via the WeakRegistry mechanism every time one or more weak arrays get finalized, only the weak arrays that have actually lost a reference will get finalized. Alas, for reasons I don't understand, this broke the file system. I would use the system for some minutes and then suddenly source code viewed in the browser would be decompiled, since the source file had closed. Alas I don't have time to look at this now, but I do think very much it is the right way to go. Right now I'm busy with a long list of dependent changes that I must get fixed to make progress. So for now I can't help other than to e.g. mail you, or post, my prototype finalization changes and have you look at those and see if you can get them working in Pharo. I;m also interested in getting anyone else who would like to help involved. I am so overloaded right now it's not funny [right, ed :-) "Get some rest. If you don't have your health, you don't have anything". Count Rugen]

thanks,
Esteban

_,,,^..^,,,_
best, Eliot

johnmci

Re: Move to Spur: how can I change WeakRegistry packages?

In reply to this post by Eliot Miranda-2

The file system gets broken likely because it uses weak logic to ensure the unix file handle gets closed when the referencing squeak file handle gets GC.

That speaks to the entity being GCed too early? Or something wrong in the squeak side finalization that makes the sources file object close it's file handle. ?

Note also if a file creation fails due to out of file handles the ancient logic there would do a full GC, hoping that there was zombie files/sockets open so that finalization would free up operating system file handles so a retry would work.

On Wed, Sep 30, 2015 at 10:15 AM, Eliot Miranda <[hidden email]> wrote:

Hi Esteban,

On Wed, Sep 30, 2015 at 5:39 AM, Esteban Lorenzano <[hidden email]> wrote:

Hi,

I need to change the WeakRegistry, to adapt Pharo to finalisation on Spur and I wanted to “star over”, injecting into Pharo the packages Collections-Weak and System-Finalizer from Squeak.
Now, that breaks everything, of course :)

So, I wonder if there is a clean way to do this?

That's a great question. A few weeks ago I implemented the Ephemeron class for Squeak (I haven't committed it yet) and set about changing the finalization scheme to support both ephemerons and non-weak-registry-based weak array finalization. The latter is where the weak array that needs finalizing is added to the same VM queue that an ephemeron that needs finalizing gets added to. So instead of all WeakArrays getting finalized via the WeakRegistry mechanism every time one or more weak arrays get finalized, only the weak arrays that have actually lost a reference will get finalized. Alas, for reasons I don't understand, this broke the file system. I would use the system for some minutes and then suddenly source code viewed in the browser would be decompiled, since the source file had closed. Alas I don't have time to look at this now, but I do think very much it is the right way to go. Right now I'm busy with a long list of dependent changes that I must get fixed to make progress. So for now I can't help other than to e.g. mail you, or post, my prototype finalization changes and have you look at those and see if you can get them working in Pharo. I;m also interested in getting anyone else who would like to help involved. I am so overloaded right now it's not funny [right, ed :-) "Get some rest. If you don't have your health, you don't have anything". Count Rugen]

thanks,
Esteban

_,,,^..^,,,_
best, Eliot

===========================================================================
John M. McIntosh. Corporate Smalltalk Consulting Ltd https://www.linkedin.com/in/smalltalk
===========================================================================

Eliot Miranda-2

Re: Move to Spur: how can I change WeakRegistry packages?

In reply to this post by EstebanLM

Hi Esteban,

so find the changes attached. To enable them (in a VMMaker.oscog-eem.1446/r3429 or later VM) you need to do SmalltalkImage current supportsQueueingFinalization: true first. I would save and restart the test image you do this in.

On Wed, Sep 30, 2015 at 10:23 AM, Esteban Lorenzano <[hidden email]> wrote:

Hi, Eliot

So, I was wanting to adopt Squeak version (since our version are working with some Igor changes that does not apply anymore…)
This started because in my FFI-NB stuff, I realised finalisers are not called at all, and I assumed it was because of the different versions.

Right now, I’m struggling to inject Squeak sources into Pharo without break everything in the middle :(

Since I cannot continue without this working, and I cannot finish migration of Pharo to spur without a FFI-NB implementation, I’m willing to spend all time I can on it.
So… yes please, send me your prototype and I will try to understand what is missing :)

Esteban

On 30 Sep 2015, at 19:15, Eliot Miranda <[hidden email]> wrote:

Hi Esteban,

On Wed, Sep 30, 2015 at 5:39 AM, Esteban Lorenzano <[hidden email]> wrote:

Hi,

I need to change the WeakRegistry, to adapt Pharo to finalisation on Spur and I wanted to “star over”, injecting into Pharo the packages Collections-Weak and System-Finalizer from Squeak.
Now, that breaks everything, of course :)

So, I wonder if there is a clean way to do this?

That's a great question. A few weeks ago I implemented the Ephemeron class for Squeak (I haven't committed it yet) and set about changing the finalization scheme to support both ephemerons and non-weak-registry-based weak array finalization. The latter is where the weak array that needs finalizing is added to the same VM queue that an ephemeron that needs finalizing gets added to. So instead of all WeakArrays getting finalized via the WeakRegistry mechanism every time one or more weak arrays get finalized, only the weak arrays that have actually lost a reference will get finalized. Alas, for reasons I don't understand, this broke the file system. I would use the system for some minutes and then suddenly source code viewed in the browser would be decompiled, since the source file had closed. Alas I don't have time to look at this now, but I do think very much it is the right way to go. Right now I'm busy with a long list of dependent changes that I must get fixed to make progress. So for now I can't help other than to e.g. mail you, or post, my prototype finalization changes and have you look at those and see if you can get them working in Pharo. I;m also interested in getting anyone else who would like to help involved. I am so overloaded right now it's not funny [right, ed :-) "Get some rest. If you don't have your health, you don't have anything". Count Rugen]
thanks,
Esteban

_,,,^..^,,,_
best, Eliot

_,,,^..^,,,_

best, Eliot

WeakArray newFinalization methods.st (6K) Download Attachment

Eliot Miranda-2

Re: Move to Spur: how can I change WeakRegistry packages?

In reply to this post by johnmci

Hi John,

On Wed, Sep 30, 2015 at 10:30 AM, John McIntosh <[hidden email]> wrote:

The file system gets broken likely because it uses weak logic to ensure the unix file handle gets closed when the referencing squeak file handle gets GC.
That speaks to the entity being GCed too early? Or something wrong in the squeak side finalization that makes the sources file object close it's file handle. ?

Note also if a file creation fails due to out of file handles the ancient logic there would do a full GC, hoping that there was zombie files/sockets open so that finalization would free up operating system file handles so a retry would work.

Right, and right. But I tried for quite a few hours to put a breakpoint in the VM to find out precisely when the sources file got closed so I could diagnose it properly but failed. So I'm leaving it for when I have some time. Volunteers welcome.

On Wed, Sep 30, 2015 at 10:15 AM, Eliot Miranda <[hidden email]> wrote:

Hi Esteban,

On Wed, Sep 30, 2015 at 5:39 AM, Esteban Lorenzano <[hidden email]> wrote:

Hi,

I need to change the WeakRegistry, to adapt Pharo to finalisation on Spur and I wanted to “star over”, injecting into Pharo the packages Collections-Weak and System-Finalizer from Squeak.
Now, that breaks everything, of course :)

So, I wonder if there is a clean way to do this?

That's a great question. A few weeks ago I implemented the Ephemeron class for Squeak (I haven't committed it yet) and set about changing the finalization scheme to support both ephemerons and non-weak-registry-based weak array finalization. The latter is where the weak array that needs finalizing is added to the same VM queue that an ephemeron that needs finalizing gets added to. So instead of all WeakArrays getting finalized via the WeakRegistry mechanism every time one or more weak arrays get finalized, only the weak arrays that have actually lost a reference will get finalized. Alas, for reasons I don't understand, this broke the file system. I would use the system for some minutes and then suddenly source code viewed in the browser would be decompiled, since the source file had closed. Alas I don't have time to look at this now, but I do think very much it is the right way to go. Right now I'm busy with a long list of dependent changes that I must get fixed to make progress. So for now I can't help other than to e.g. mail you, or post, my prototype finalization changes and have you look at those and see if you can get them working in Pharo. I;m also interested in getting anyone else who would like to help involved. I am so overloaded right now it's not funny [right, ed :-) "Get some rest. If you don't have your health, you don't have anything". Count Rugen]

thanks,
Esteban

_,,,^..^,,,_
best, Eliot

--
===========================================================================
John M. McIntosh. Corporate Smalltalk Consulting Ltd https://www.linkedin.com/in/smalltalk
===========================================================================

_,,,^..^,,,_

best, Eliot