Weak finalization enh. are still not integrated into VM(s)

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

Weak finalization enh. are still not integrated into VM(s)

Igor Stasenko
 
Hello guys,

i'd like to remind you that this stuff is not yet made own way to VM and images.

Magma suffers badly from old finalization implementation, and as a
consequence it losing in speed competition
comparing to other OODBs.
I'd really like to see this enhancement to be integrated to both
Squeak VM and Cog VM.

My quick analyzis shows that Squeak 4.2 image contains some things,
which needed:

        "Weak reference finalization"
        newArray at: 56 put: (self at: #WeakFinalizer ifAbsent: []).

while Pharo still not.

I'd like to remind you that VM-side changes are really simple and
backwards compatible (given the right size of special objects array).
Once we will have new VMs, supporting this feature , we could start
integrating the image-side part of it.

Please, inform me, what i can do to help closing this issue.


--
Best regards,
Igor Stasenko AKA sig.
Reply | Threaded
Open this post in threaded view
|

Re: Weak finalization enh. are still not integrated into VM(s)

Eliot Miranda-2
 


On Mon, Sep 20, 2010 at 4:02 PM, Igor Stasenko <[hidden email]> wrote:

Hello guys,

i'd like to remind you that this stuff is not yet made own way to VM and images.

Magma suffers badly from old finalization implementation, and as a
consequence it losing in speed competition
comparing to other OODBs.
I'd really like to see this enhancement to be integrated to both
Squeak VM and Cog VM.

My quick analyzis shows that Squeak 4.2 image contains some things,
which needed:

       "Weak reference finalization"
       newArray at: 56 put: (self at: #WeakFinalizer ifAbsent: []).

while Pharo still not.

I'd like to remind you that VM-side changes are really simple and
backwards compatible (given the right size of special objects array).
Once we will have new VMs, supporting this feature , we could start
integrating the image-side part of it.

Please, inform me, what i can do to help closing this issue.

you can either send me a change set that is w.r.t. to a Cog version or save a Monticello version that is a delta from Cog.  I also need image level test code to pound on this.

best,
Eliot



--
Best regards,
Igor Stasenko AKA sig.

Reply | Threaded
Open this post in threaded view
|

Re: Weak finalization enh. are still not integrated into VM(s)

Igor Stasenko
 
On 21 September 2010 02:08, Eliot Miranda <[hidden email]> wrote:

>
>
>
> On Mon, Sep 20, 2010 at 4:02 PM, Igor Stasenko <[hidden email]> wrote:
>>
>> Hello guys,
>>
>> i'd like to remind you that this stuff is not yet made own way to VM and images.
>>
>> Magma suffers badly from old finalization implementation, and as a
>> consequence it losing in speed competition
>> comparing to other OODBs.
>> I'd really like to see this enhancement to be integrated to both
>> Squeak VM and Cog VM.
>>
>> My quick analyzis shows that Squeak 4.2 image contains some things,
>> which needed:
>>
>>        "Weak reference finalization"
>>        newArray at: 56 put: (self at: #WeakFinalizer ifAbsent: []).
>>
>> while Pharo still not.
>>
>> I'd like to remind you that VM-side changes are really simple and
>> backwards compatible (given the right size of special objects array).
>> Once we will have new VMs, supporting this feature , we could start
>> integrating the image-side part of it.
>>
>> Please, inform me, what i can do to help closing this issue.
>
> you can either send me a change set that is w.r.t. to a Cog version or save a Monticello version that is a delta from Cog.  I also need image level test code to pound on this.

The issue #7473 are dedicated for both vm-side changes and
image-side changes.

The image-side scripts are:

- Weak-finalizers-classes.2.cs
- new-finalization-auto.2.cs

install them in the above order, then run

WeakArray restartFinalizationProcess

I added the test case for new kind of registry, and all tests which
are green for old registry
also green for a new one.
The new registry are automatically detects if VM supports new
finalization or not,
and switching to suitable finalization logic  (it takes a single GC
in order to detect it).


The last thing is #recreateSpecialObjectsArray  [2].
A new entry is already added in Squeak, thanks Levente,
except that naming is wrong, and should refer to #WeakFinalizationList,
instead of  #WeakFinalizer , i.e.

        "Weak reference finalization"
        newArray at: 56 put: (self at: #WeakFinalizationList ifAbsent: []).


The VM-side changes is in file
 weak-finalizers-VM.1.cs ,
 but they are for Squeak VM, and a bit outdated, since index of
special object is wrong
(obviously, should be in sync with one in spl objs array).

There are two methods, which don't require changes:

ObjectMemory>>finalizeReference:
and
ObjectMemory>>weakFinalizerCheck:

And on a class side, the #intitializeSpecialObjectIndices, should use
a correct index for
WeakFinalizer class var, i.e.

WeakFinalizer := 55.


For Cog VM, i made everything fixed (see changesets in attachments).
I was able to build new Cog VM and it works fine, reporting:

WeakFinalizationRegistry hasNewFinalization  ==> true

after starting an image under this VM.

1. http://bugs.squeak.org/view.php?id=7473
2. http://bugs.squeak.org/view.php?id=7525

> best,
> Eliot

--
Best regards,
Igor Stasenko AKA sig.

Cog-weakfinalizers.1.cs (14K) Download Attachment
splObjsArray.1.cs (7K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Weak finalization enh. are still not integrated into VM(s)

Igor Stasenko
 
I am also uploaded a slice of system changes with new classes & rest
of stuff into inbox:


CollectionsTests-Igor.Stasenko.169
Collections-Igor.Stasenko.370
Kernel-Igor.Stasenko.478
System-Igor.Stasenko.358

note, that despite of these changes, system are still using old
WeakRegistry instance,
and obviously 'WeakRegistry default' needs to be modified to be
replaced with new functionality.


--
Best regards,
Igor Stasenko AKA sig.
Reply | Threaded
Open this post in threaded view
|

Re: Weak finalization enh. are still not integrated into VM(s)

David T. Lewis
In reply to this post by Igor Stasenko
 
On Tue, Sep 21, 2010 at 03:17:15AM +0300, Igor Stasenko wrote:

>  
> On 21 September 2010 02:08, Eliot Miranda <[hidden email]> wrote:
> >
> >
> >
> > On Mon, Sep 20, 2010 at 4:02 PM, Igor Stasenko <[hidden email]> wrote:
> >>
> >> Hello guys,
> >>
> >> i'd like to remind you that this stuff is not yet made own way to VM and images.
> >>
> >> Magma suffers badly from old finalization implementation, and as a
> >> consequence it losing in speed competition
> >> comparing to other OODBs.
> >> I'd really like to see this enhancement to be integrated to both
> >> Squeak VM and Cog VM.
> >>
> >> My quick analyzis shows that Squeak 4.2 image contains some things,
> >> which needed:
> >>
> >> ?? ?? ?? ??"Weak reference finalization"
> >> ?? ?? ?? ??newArray at: 56 put: (self at: #WeakFinalizer ifAbsent: []).
> >>
> >> while Pharo still not.
> >>
> >> I'd like to remind you that VM-side changes are really simple and
> >> backwards compatible (given the right size of special objects array).
> >> Once we will have new VMs, supporting this feature , we could start
> >> integrating the image-side part of it.
> >>
> >> Please, inform me, what i can do to help closing this issue.
> >
> > you can either send me a change set that is w.r.t. to a Cog version or save a Monticello version that is a delta from Cog. ??I also need image level test code to pound on this.
>
>
> The issue #7473 are dedicated for both vm-side changes and
> image-side changes.

The VM side changes should be in place as of VMMaker-dtl.175, version
4.2.2 (sorry I forgot to update the Mantis issue when I did this). That
means that the changes are in the VMMaker package, but are not yet in
published VM releases (e.g. the Unix VM Squeak-4.0.3.2202 was built
from VMM version 4.0.3, prior to 4.2.2).

Eliot - the ObjectMemory>>initializeSpecialObjectIndices in VMMaker-dtl.175
ends as follows, which is intended to be consistent with Cog and Alien
requirements, and should match the #recreateSpecialObjectsArray that
Igor provided.

        SelectorAboutToReturn := 48.
        SelectorRunWithIn := 49.
        SelectorAttemptToAssign := 50.

        "PrimErrTableIndex := 51. in Interpreter class>>initializePrimitiveErrorCodes"

        ClassAlien := 52.
        InvokeCallbackSelector := 53.
        ClassUnsafeAlien := 54.

        ClassWeakFinalizer := 55

Dave

>
> The image-side scripts are:
>
> - Weak-finalizers-classes.2.cs
> - new-finalization-auto.2.cs
>
> install them in the above order, then run
>
> WeakArray restartFinalizationProcess
>
> I added the test case for new kind of registry, and all tests which
> are green for old registry
> also green for a new one.
> The new registry are automatically detects if VM supports new
> finalization or not,
> and switching to suitable finalization logic  (it takes a single GC
> in order to detect it).
>
>
> The last thing is #recreateSpecialObjectsArray  [2].
> A new entry is already added in Squeak, thanks Levente,
> except that naming is wrong, and should refer to #WeakFinalizationList,
> instead of  #WeakFinalizer , i.e.
>
> "Weak reference finalization"
> newArray at: 56 put: (self at: #WeakFinalizationList ifAbsent: []).
>
>
> The VM-side changes is in file
>  weak-finalizers-VM.1.cs ,
>  but they are for Squeak VM, and a bit outdated, since index of
> special object is wrong
> (obviously, should be in sync with one in spl objs array).
>
> There are two methods, which don't require changes:
>
> ObjectMemory>>finalizeReference:
> and
> ObjectMemory>>weakFinalizerCheck:
>
> And on a class side, the #intitializeSpecialObjectIndices, should use
> a correct index for
> WeakFinalizer class var, i.e.
>
> WeakFinalizer := 55.
>
>
> For Cog VM, i made everything fixed (see changesets in attachments).
> I was able to build new Cog VM and it works fine, reporting:
>
> WeakFinalizationRegistry hasNewFinalization  ==> true
>
> after starting an image under this VM.
>
> 1. http://bugs.squeak.org/view.php?id=7473
> 2. http://bugs.squeak.org/view.php?id=7525
>
> > best,
> > Eliot
>
> --
> Best regards,
> Igor Stasenko AKA sig.

Reply | Threaded
Open this post in threaded view
|

Re: Weak finalization enh. are still not integrated into VM(s)

David T. Lewis
In reply to this post by Igor Stasenko
 
On Tue, Sep 21, 2010 at 03:28:35AM +0300, Igor Stasenko wrote:

>  
> I am also uploaded a slice of system changes with new classes & rest
> of stuff into inbox:
>
>
> CollectionsTests-Igor.Stasenko.169
> Collections-Igor.Stasenko.370
> Kernel-Igor.Stasenko.478
> System-Igor.Stasenko.358
>
> note, that despite of these changes, system are still using old
> WeakRegistry instance,
> and obviously 'WeakRegistry default' needs to be modified to be
> replaced with new functionality.

These inbox changes are out of date with respect to the trunk, so
would need to be merged with care. But aside from that, is there
any reason that your changes should *not* be moved into trunk now?
If no objections, then we should include them now.

I understand that the VM changes are not yet available in the
official VMs, but this should be corrected the next time that new
VMs are distributed.

Dave

Reply | Threaded
Open this post in threaded view
|

Re: Weak finalization enh. are still not integrated into VM(s)

Igor Stasenko
 
On 21 September 2010 06:20, David T. Lewis <[hidden email]> wrote:

>
> On Tue, Sep 21, 2010 at 03:28:35AM +0300, Igor Stasenko wrote:
>>
>> I am also uploaded a slice of system changes with new classes & rest
>> of stuff into inbox:
>>
>>
>> CollectionsTests-Igor.Stasenko.169
>> Collections-Igor.Stasenko.370
>> Kernel-Igor.Stasenko.478
>> System-Igor.Stasenko.358
>>
>> note, that despite of these changes, system are still using old
>> WeakRegistry instance,
>> and obviously 'WeakRegistry default' needs to be modified to be
>> replaced with new functionality.
>
> These inbox changes are out of date with respect to the trunk, so
> would need to be merged with care.

Yeah, i did not updated the image before applying changesets.

> But aside from that, is there
> any reason that your changes should *not* be moved into trunk now?
> If no objections, then we should include them now.
>

There are a couple of doits, which should be invoked after loading changes:

WeakArray restartFinalizationProcess
and of couse
Smalltalk recreateSpecialObjectsArray

Btw, i found that in image i was using,
Smalltalk specialObjectsArray size
was 50
while in #recreateSpecialObjectsArray method there was already 55 entries
(and with weak finalization it should be 56).

> I understand that the VM changes are not yet available in the
> official VMs, but this should be corrected the next time that new
> VMs are distributed.
>
> Dave
>
>
Thanks, David.


--
Best regards,
Igor Stasenko AKA sig.
Reply | Threaded
Open this post in threaded view
|

Re: Weak finalization enh. are still not integrated into VM(s)

Casey Ransberger-2
 
Guessing this is why WeakFinalizersTest>>testNewFinalizationSupported is failing. Someone asked me to make a new trunk image; is it safe to do that with this test erroring out? Sounds like it is.

On Tue, Sep 21, 2010 at 6:22 AM, Igor Stasenko <[hidden email]> wrote:

On 21 September 2010 06:20, David T. Lewis <[hidden email]> wrote:
>
> On Tue, Sep 21, 2010 at 03:28:35AM +0300, Igor Stasenko wrote:
>>
>> I am also uploaded a slice of system changes with new classes & rest
>> of stuff into inbox:
>>
>>
>> CollectionsTests-Igor.Stasenko.169
>> Collections-Igor.Stasenko.370
>> Kernel-Igor.Stasenko.478
>> System-Igor.Stasenko.358
>>
>> note, that despite of these changes, system are still using old
>> WeakRegistry instance,
>> and obviously 'WeakRegistry default' needs to be modified to be
>> replaced with new functionality.
>
> These inbox changes are out of date with respect to the trunk, so
> would need to be merged with care.

Yeah, i did not updated the image before applying changesets.

> But aside from that, is there
> any reason that your changes should *not* be moved into trunk now?
> If no objections, then we should include them now.
>

There are a couple of doits, which should be invoked after loading changes:

WeakArray restartFinalizationProcess
and of couse
Smalltalk recreateSpecialObjectsArray

Btw, i found that in image i was using,
Smalltalk specialObjectsArray size
was 50
while in #recreateSpecialObjectsArray method there was already 55 entries
(and with weak finalization it should be 56).

> I understand that the VM changes are not yet available in the
> official VMs, but this should be corrected the next time that new
> VMs are distributed.
>
> Dave
>
>
Thanks, David.


--
Best regards,
Igor Stasenko AKA sig.



--
Casey Ransberger
Reply | Threaded
Open this post in threaded view
|

Re: Weak finalization enh. are still not integrated into VM(s)

Igor Stasenko
 
On 22 September 2010 04:09, Casey Ransberger <[hidden email]> wrote:
>
> Guessing this is why WeakFinalizersTest>>testNewFinalizationSupported is failing. Someone asked me to make a new trunk image; is it safe to do that with this test erroring out? Sounds like it is.

It passing on my freshly built Cog VM with finalization.
This test is written specifically to tell if VM is supporting it or not.

> On Tue, Sep 21, 2010 at 6:22 AM, Igor Stasenko <[hidden email]> wrote:
>>
>> On 21 September 2010 06:20, David T. Lewis <[hidden email]> wrote:
>> >
>> > On Tue, Sep 21, 2010 at 03:28:35AM +0300, Igor Stasenko wrote:
>> >>
>> >> I am also uploaded a slice of system changes with new classes & rest
>> >> of stuff into inbox:
>> >>
>> >>
>> >> CollectionsTests-Igor.Stasenko.169
>> >> Collections-Igor.Stasenko.370
>> >> Kernel-Igor.Stasenko.478
>> >> System-Igor.Stasenko.358
>> >>
>> >> note, that despite of these changes, system are still using old
>> >> WeakRegistry instance,
>> >> and obviously 'WeakRegistry default' needs to be modified to be
>> >> replaced with new functionality.
>> >
>> > These inbox changes are out of date with respect to the trunk, so
>> > would need to be merged with care.
>>
>> Yeah, i did not updated the image before applying changesets.
>>
>> > But aside from that, is there
>> > any reason that your changes should *not* be moved into trunk now?
>> > If no objections, then we should include them now.
>> >
>>
>> There are a couple of doits, which should be invoked after loading changes:
>>
>> WeakArray restartFinalizationProcess
>> and of couse
>> Smalltalk recreateSpecialObjectsArray
>>
>> Btw, i found that in image i was using,
>> Smalltalk specialObjectsArray size
>> was 50
>> while in #recreateSpecialObjectsArray method there was already 55 entries
>> (and with weak finalization it should be 56).
>>
>> > I understand that the VM changes are not yet available in the
>> > official VMs, but this should be corrected the next time that new
>> > VMs are distributed.
>> >
>> > Dave
>> >
>> >
>> Thanks, David.
>>
>>
>> --
>> Best regards,
>> Igor Stasenko AKA sig.
>
>
>
> --
> Casey Ransberger
>
>



--
Best regards,
Igor Stasenko AKA sig.
Reply | Threaded
Open this post in threaded view
|

Re: Weak finalization enh. are still not integrated into VM(s)

David T. Lewis
In reply to this post by Igor Stasenko
 
On Tue, Sep 21, 2010 at 04:22:22PM +0300, Igor Stasenko wrote:

>  
> On 21 September 2010 06:20, David T. Lewis <[hidden email]> wrote:
> >
> > On Tue, Sep 21, 2010 at 03:28:35AM +0300, Igor Stasenko wrote:
> >>
> >> I am also uploaded a slice of system changes with new classes & rest
> >> of stuff into inbox:
> >>
> >>
> >> CollectionsTests-Igor.Stasenko.169
> >> Collections-Igor.Stasenko.370
> >> Kernel-Igor.Stasenko.478
> >> System-Igor.Stasenko.358
> >>
> >> note, that despite of these changes, system are still using old
> >> WeakRegistry instance,
> >> and obviously 'WeakRegistry default' needs to be modified to be
> >> replaced with new functionality.
> >
> > These inbox changes are out of date with respect to the trunk, so
> > would need to be merged with care.
>
> Yeah, i did not updated the image before applying changesets.
>
> > But aside from that, is there
> > any reason that your changes should *not* be moved into trunk now?
> > If no objections, then we should include them now.
> >
>
> There are a couple of doits, which should be invoked after loading changes:
>
> WeakArray restartFinalizationProcess
> and of couse
> Smalltalk recreateSpecialObjectsArray
>
> Btw, i found that in image i was using,
> Smalltalk specialObjectsArray size
> was 50
> while in #recreateSpecialObjectsArray method there was already 55 entries
> (and with weak finalization it should be 56).

This should already be updated in trunk:

        Name: System-dtl.333
        Author: dtl
        Time: 25 May 2010, 9:00:22.752 pm
        Ancestors: System-spd.332
        Update SmalltalkImage>>recreateSpecialObjectsArray for support of Alien and improved
        finalization (Mantis 7473). Corresponding VM support is in SqS/VMMaker-dtl.175.

So after a #recreateSpecialObjectsArray the size is 56, and the last slot
is used for WeakFinalizer if present in the image.

I think that everything is ready for your WeakFinalizer updates. Unless
there are any other issues, please just go ahead and commit them to trunk.

:)

Dave

>
> > I understand that the VM changes are not yet available in the
> > official VMs, but this should be corrected the next time that new
> > VMs are distributed.
> >
> > Dave
> >
> >
> Thanks, David.
>
>
> --
> Best regards,
> Igor Stasenko AKA sig.