orphaned classes do not disappear

Hi there,

I am working on preparing a GS 2.4.4.1 extent for upgrade to GS 3.2
For that, I am cleaning the repo such that no orphaned class instances still exist.

I hit an issue because I find orphaned classes that are not GC’ed for which no instances still exist.
When I ask the reference path to the class object, it tells me it is being referenced by its metaclass.
However, when I ask for the reference path to the metaclass object, I get a ‘false’ response.

Is there anything I can do to make sure these instances (of MetaClass) are removed (or otherwise find out why they are not GC’ed) ?

thanks
Johan

For reference, this is the script to find orphaned classes:
 | classes orphans |
    classes := IdentitySet new.
    System myUserProfile symbolList do: [:symbolDictionary |
      symbolDictionary valuesDo: [:global |
              global isBehavior ifTrue: [ classes add: global ]]].
    System commitTransaction.
    orphans :=  (Metaclass allInstances collect: [:mClass | mClass theNonMetaClass ])
                              select: [:class | (classes includes: class) not ].
_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

Also:

I already executed the following expressions I picked up from earlier discussions and that fix some memory leaks:


GsDeployer cleanClassHistory.
  MCDefinition clearInstances.
  MCMethodDefinition resetCachedDefinitions.
        UndefinedSymbols removeAll.
  MethodVersionHistory reset.
  "MetacelloConfigurationResource reset."
  UserGlobals removeKey: #GsDeployerBar_Instances ifAbsent: [].
  SystemOrganizer resetSystemOrganization.
  UndefinedSymbolsTest new cleanUndefinedSymbols.

  System myUserProfile symbolList do: [:symbolDictionary |
        symbolDictionary valuesDo: [:global |
          global isBehavior ifTrue: [ | desc |
            (desc := global instVarNamed: 'description' ) notNil
              ifTrue: [
                desc itsClass ~~ global
                  ifTrue: [ global description itsClass: global ]]]]].


_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

Are the orphaned classes still in the class history of current class
version?

Have you tried calling
Repository>>findReferencePathToObjs:limitObjArray:findAllRefs:printToLog:
with an empty limitObjArray? I think that the default listReferences
code uses something similar to Repository>>buildLimitSetForRefPathScan
as it's list of repository roots, which are ignored for purposes of list
references, but probably still useful when hunting down stale metaclass
refs.

Dale

On 3/8/17 4:01 AM, Johan Brichau via Glass wrote:
_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

Hi Dale,

> On 10 Mar 2017, at 16:25, Dale Henrichs via Glass <[hidden email]> wrote:
>
> Are the orphaned classes still in the class history of current class version?

No, the classHistories all contain only the current version of the class.
It seems the classHistory object is the same as the one referenced by the current version.

> Have you tried calling Repository>>findReferencePathToObjs:limitObjArray:findAllRefs:printToLog: with an empty limitObjArray? I think that the default listReferences code uses something similar to Repository>>buildLimitSetForRefPathScan as it's list of repository roots, which are ignored for purposes of list references, but probably still useful when hunting down stale metaclass refs.

I tried this but you cannot pass an empty array (there is an explicit check in the method).
However, I tried passing an array { #none } and still did not find any reference paths for a single MetaClass instance.
When I try passing the entire list of metaclasses, I hit an error:

The object with object ID 1339726081 is corrupt. Reason: ‘Error no limit oop found in last chain link for search oop 1339726081
The object in question is such a MetaClass instance.

Johan

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

On 03/16/2017 09:11 AM, Johan Brichau via Glass wrote:
> Hi Dale,
>
>> On 10 Mar 2017, at 16:25, Dale Henrichs via Glass <[hidden email]> wrote:
>>
>> Are the orphaned classes still in the class history of current class version?
> No, the classHistories all contain only the current version of the class.
> It seems the classHistory object is the same as the one referenced by the current version.
okay..
>
>> Have you tried calling Repository>>findReferencePathToObjs:limitObjArray:findAllRefs:printToLog: with an empty limitObjArray? I think that the default listReferences code uses something similar to Repository>>buildLimitSetForRefPathScan as it's list of repository roots, which are ignored for purposes of list references, but probably still useful when hunting down stale metaclass refs.
> I tried this but you cannot pass an empty array (there is an explicit check in the method).
> However, I tried passing an array { #none } and still did not find any reference paths for a single MetaClass instance.
> When I try passing the entire list of metaclasses, I hit an error:
>
> The object with object ID 1339726081 is corrupt. Reason: ‘Error no limit oop found in last chain link for search oop 1339726081
> The object in question is such a MetaClass instance.
Hmmm, I know that over the years (and especially recently) we have done
work to improve the find reference code. I am not intimately familiar
with with the bugs that have been fixed over the years, so it would be
possible that we have bugs/features that are not finding the reference
for your object ...

It is tempting to suggest that you upgrade your repository to 3.3.3 and
use the find references code for that version --- in this case you
wouldn't need to load code, but could just run the basic upgradeImage
script and then use topaz to run reference scans ...

Is the object that you are searching for holding onto a lot other state
or otherwise preventing you from upgrading? If it is just an annoyance?

If you consider it a real problem and you are willing to share a copy of
your db, we'd be willing to try to figure out what's going on --- in our
case, we'd more than likely upgrade the db ourselves and debug the issue
there ... if it is still an issue:)

Dale
_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

Hi Dale,
_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

Dale,
_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

Ah, very interesting ... I think that in 3.3.3 we just might have been able to find those references .... IIRC:) ... I know that we have focused in this area for both 3.3.3 and 3.4 to try to improve things  ... I've got a work in progress project called Object explorer[1] where I'm building some tools for finding references and tracking down object paths ... the project is aimed at 3.4, because we've added some new support for finding parents of objects (efficiently) and this provides a way to pretty much guarantee that you can find the references that are keeping an object alive ... The object explorer is an interactive tool ... finding all reference paths automatically without getting someone with domain knowledge involved  can be difficult because recursive data structures cause big time memory/performance issues ...

Dale

[1] https://github.com/dalehenrich/obex

Dale,

I discovered I had GsBackupScan64 loaded in this extent and there is a class variable ‘Classes’ that keeps a mapping oop -> class.

Needless to say… I cannot even remember I loaded it into this extent :) This was probably done in the very early days of Yesplan.

Too bad those references did not show up when scanning the references.

I’m down to just a very short list of classes now. So I’m guessing I will be out of the woods soon on this one.

Johan

On 17 Mar 2017, at 17:10, Johan Brichau <[hidden email]> wrote:

Hi Dale,

On 16 Mar 2017, at 23:14, Dale Henrichs via Glass <[hidden email]> wrote:

Is the object that you are searching for holding onto a lot other state or otherwise preventing you from upgrading? If it is just an annoyance?

As there are no instances of these classes anymore, it’s an annoyance, as I’m wondering what is referencing these classes.

I just did a bit more poking and when I emptied the method dictionaries for these classes, many of them were garbage collected.

It’s not just our own classes, but I also see a lot of Metacello* and XML* class(-versions) in that list.

I will probably just try what you suggested and see what info I get in 3.3.3 

If you consider it a real problem and you are willing to share a copy of your db, we'd be willing to try to figure out what's going on --- in our case, we'd more than likely upgrade the db ourselves and debug the issue there ... if it is still an issue:)

Thx, if I’m still lost in a week from now, I might just send in a ticket for help :)

Johan

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass