instances of old class versions

Hi,

We encountered a strange situation today: we encountered an instance of an 'old' class in one of our repositories. This old class did not appear anymore in the class history of its current version (which is normal: I'm using GsDeployer>>doBulkMigrate) and all other instances (thousands) were correctly migrated.

So, my question: is there a way to find out if there are still instances of "old" classes floating around in the system?

I know that we can ask for the classhistory and check the old class versions, but this old class was no longer found in the classhistory of its current version. Just to illustrate, supposing that I'm talking about SomeClass, that means the following ('self' is the instance):

self class == SomeClass
-> false

SomeClass classHistory includes: self class
-> false

SomeClass allInstances includes: self
-> false

I corrected the problem using the #migrateInstancesTo: on this instance, but now I'm left wondering if there might be others and how I could try to find those.

The other question would, of course, be: "how did this happen"? I'm guessing that doing a live update and migrating instances while still serving Seaside requests may lead to this problem?

thanks for any suggestions!
Johan

Johan,

The existence of an instance of an old class that is not in class history means that the instance slipped through the cracks when the class histories were cleaned up...

Which then implies that the instance could have been created while the bulk migrate was executing ...

Which implies that it might make sense to defer the clean up of class histories for such time when all of the vms have updated to a view containing the latest class version.

Off the top of my head this could be accomplished by inserting a step in the bulk migrate process where a commit is performed and then staying in transaction until the current session is the oldest session with the commit record...

Have to ponder a clean way of doing this in a timely fashion, since it probably would involve another session doing commits to push the commit record backlog up to the point where the stone starts sending out commit aborts...

To find instances of obsolete classes I devised the following three-step process:

  1. Clear out the caches that might be harboring references to obsolete classes or otherwise keeping
     obsolete instances alive:

     GsDeployer cleanClassHistory.
     MCDefinition clearInstances.
     MCMethodDefinition resetCachedDefinitions.
     MethodVersionHistory reset.
     MetacelloConfigurationResource reset.
     UserGlobals removeKey: #GsDeployerBar_Instances ifAbsent: [].
     SystemOrganizer resetSystemOrganization.
  2. Log out and log back in, then run an MFC
  3. Wait a sufficient period for dead objects to be processed and then execute the following:
      | classes orphans |
      classes := IdentitySet new.
      System myUserProfile symbolList do: [:symbolDictionary |
        symbolDictionary valuesDo: [:global |
                global isBehavior ifTrue: [ classes add: global ]]].
      System commitTransaction.
      orphans :=  (Metaclass allInstances collect: [:mClass | mClass theNonMetaClass ])
                                select: [:class | (classes includes: class) not ].

The orphans will be a list of classes that are not reachable from your symbol list. The following list are system classes and can be safely ignored:

   ObsoleteGeode, LargeObjectNode, NscBagLeaf, NscSetLeaf, NscInteriorNode, NscNode

`GsDeployer cleanClassHistory` cleans up all of the class histories (reachable from your symbol list), so the remaining classes aren't in the class history and are being kept alive for some reason and are instances of those classes are candidates for migration ...

This algorithm doesn't take into account multiple users (SystemUser is accounted for) nor does it take into account GsPackageLibraries that have not been loaded (the classes that implement Monticello for the base in GemStone/S 3.0.1 are an example), but this should be sufficient for 2.4.4.1 users. I'll update the algorithm to be more complete at a later date ...

Dale




----- Original Message -----
| From: "Johan Brichau" <[hidden email]>
| To: "GemStone Seaside beta discussion" <[hidden email]>
| Sent: Wednesday, November 23, 2011 10:45:44 AM
| Subject: [GS/SS Beta] instances of old class versions
|
| Hi,
|
| We encountered a strange situation today: we encountered an instance
| of an 'old' class in one of our repositories. This old class did not
| appear anymore in the class history of its current version (which is
| normal: I'm using GsDeployer>>doBulkMigrate) and all other instances
| (thousands) were correctly migrated.
|
| So, my question: is there a way to find out if there are still
| instances of "old" classes floating around in the system?
|
| I know that we can ask for the classhistory and check the old class
| versions, but this old class was no longer found in the classhistory
| of its current version. Just to illustrate, supposing that I'm
| talking about SomeClass, that means the following ('self' is the
| instance):
|
| self class == SomeClass
| -> false
|
| SomeClass classHistory includes: self class
| -> false
|
| SomeClass allInstances includes: self
| -> false
|
| I corrected the problem using the #migrateInstancesTo: on this
| instance, but now I'm left wondering if there might be others and
| how I could try to find those.
|
| The other question would, of course, be: "how did this happen"? I'm
| guessing that doing a live update and migrating instances while
| still serving Seaside requests may lead to this problem?
|
| thanks for any suggestions!
| Johan

On Nov 23, 2011, at 4:28 PM, Dale Henrichs wrote:

> Off the top of my head this could be accomplished by inserting a step in the bulk migrate process where a commit is performed and then staying in transaction until the current session is the oldest session with the commit record...
>
> Have to ponder a clean way of doing this in a timely fashion, since it probably would involve another session doing commits to push the commit record backlog up to the point where the stone starts sending out commit aborts...

You don't need to wait for the stone to send sigAborts. Just evaluate the following:

        System currentSessions do: [:each | System sendSigAbortToSession: each negated].

James

Hi Dale,

Coming back to this issue....

Using your script below, I found a relatively large number of 'old' classes in my repository that only seem to be referenced by their metaclass.
A search for references to find out why these classes are being hold onto always points to their metaclass and no path can be found for those metaclasses.
(I'm using 'SystemRepository findReferencePathToObject: <class object>' )

So, I'm puzzled why these classes remain in there. Are there any other ideas that pop into mind?

thanks
Johan

On 24 Nov 2011, at 01:28, Dale Henrichs wrote:

Johan,

I assume that you have already checked for instances of these classes? That would be the number one reason for the classes to hang around ...

I seem to recall some "funny business" involving meta classes keeping things alive, but I don't recall the details and the guys that would help me remember have left for the weekend... and I think that findReferencePathToObject: might also do something special with classes (like possibly ignore instances), but again I'm short on details.

I also assume that you've cleaned the class histories? If there are classes kept alive by a class history, they would show up as orphaned and the findReferencePathToObject: might now search the class history...

If these ideas don't pan out, I'll talk to the guys no Monday and see if they have any other ideas.

Dale

----- Original Message -----
| From: "Johan Brichau" <[hidden email]>
| To: "GemStone Seaside beta discussion" <[hidden email]>
| Sent: Friday, February 10, 2012 2:52:52 PM
| Subject: Re: [GS/SS Beta] instances of old class versions
|
| Hi Dale,
|
| Coming back to this issue....
|
| Using your script below, I found a relatively large number of 'old'
| classes in my repository that only seem to be referenced by their
| metaclass.
| A search for references to find out why these classes are being hold
| onto always points to their metaclass and no path can be found for
| those metaclasses.
| (I'm using 'SystemRepository findReferencePathToObject: <class
| object>' )
|
| So, I'm puzzled why these classes remain in there. Are there any
| other ideas that pop into mind?
|
| thanks
| Johan
|
| On 24 Nov 2011, at 01:28, Dale Henrichs wrote:
|
| > Johan,
| >
| > The existence of an instance of an old class that is not in class
| > history means that the instance slipped through the cracks when
| > the class histories were cleaned up...
| >
| > Which then implies that the instance could have been created while
| > the bulk migrate was executing ...
| >
| > Which implies that it might make sense to defer the clean up of
| > class histories for such time when all of the vms have updated to
| > a view containing the latest class version.
| >
| > Off the top of my head this could be accomplished by inserting a
| > step in the bulk migrate process where a commit is performed and
| > then staying in transaction until the current session is the
| > oldest session with the commit record...
| >
| > Have to ponder a clean way of doing this in a timely fashion, since
| > it probably would involve another session doing commits to push
| > the commit record backlog up to the point where the stone starts
| > sending out commit aborts...
| >
| > To find instances of obsolete classes I devised the following
| > three-step process:
| >
| >  1. Clear out the caches that might be harboring references to
| >  obsolete classes or otherwise keeping
| >     obsolete instances alive:
| >
| >     GsDeployer cleanClassHistory.
| >     MCDefinition clearInstances.
| >     MCMethodDefinition resetCachedDefinitions.
| >     MethodVersionHistory reset.
| >     MetacelloConfigurationResource reset.
| >     UserGlobals removeKey: #GsDeployerBar_Instances ifAbsent: [].
| >     SystemOrganizer resetSystemOrganization.
| >  2. Log out and log back in, then run an MFC
| >  3. Wait a sufficient period for dead objects to be processed and
| >  then execute the following:
| >      | classes orphans |
| >      classes := IdentitySet new.
| >      System myUserProfile symbolList do: [:symbolDictionary |
| > symbolDictionary valuesDo: [:global |
| > global isBehavior ifTrue: [ classes add: global ]]].
| >      System commitTransaction.
| >      orphans :=  (Metaclass allInstances collect: [:mClass | mClass
| >      theNonMetaClass ])
| > select: [:class | (classes includes: class) not ].
| >
| > The orphans will be a list of classes that are not reachable from
| > your symbol list. The following list are system classes and can be
| > safely ignored:
| >
| >   ObsoleteGeode, LargeObjectNode, NscBagLeaf, NscSetLeaf,
| >   NscInteriorNode, NscNode
| >
| > `GsDeployer cleanClassHistory` cleans up all of the class histories
| > (reachable from your symbol list), so the remaining classes aren't
| > in the class history and are being kept alive for some reason and
| > are instances of those classes are candidates for migration ...
| >
| > This algorithm doesn't take into account multiple users (SystemUser
| > is accounted for) nor does it take into account GsPackageLibraries
| > that have not been loaded (the classes that implement Monticello
| > for the base in GemStone/S 3.0.1 are an example), but this should
| > be sufficient for 2.4.4.1 users. I'll update the algorithm to be
| > more complete at a later date ...
| >
| > Dale
| >
| >
| >
| >
| > ----- Original Message -----
| > | From: "Johan Brichau" <[hidden email]>
| > | To: "GemStone Seaside beta discussion"
| > | <[hidden email]>
| > | Sent: Wednesday, November 23, 2011 10:45:44 AM
| > | Subject: [GS/SS Beta] instances of old class versions
| > |
| > | Hi,
| > |
| > | We encountered a strange situation today: we encountered an
| > | instance
| > | of an 'old' class in one of our repositories. This old class did
| > | not
| > | appear anymore in the class history of its current version (which
| > | is
| > | normal: I'm using GsDeployer>>doBulkMigrate) and all other
| > | instances
| > | (thousands) were correctly migrated.
| > |
| > | So, my question: is there a way to find out if there are still
| > | instances of "old" classes floating around in the system?
| > |
| > | I know that we can ask for the classhistory and check the old
| > | class
| > | versions, but this old class was no longer found in the
| > | classhistory
| > | of its current version. Just to illustrate, supposing that I'm
| > | talking about SomeClass, that means the following ('self' is the
| > | instance):
| > |
| > | self class == SomeClass
| > | -> false
| > |
| > | SomeClass classHistory includes: self class
| > | -> false
| > |
| > | SomeClass allInstances includes: self
| > | -> false
| > |
| > | I corrected the problem using the #migrateInstancesTo: on this
| > | instance, but now I'm left wondering if there might be others and
| > | how I could try to find those.
| > |
| > | The other question would, of course, be: "how did this happen"?
| > | I'm
| > | guessing that doing a live update and migrating instances while
| > | still serving Seaside requests may lead to this problem?
| > |
| > | thanks for any suggestions!
| > | Johan
|
|

Hi Dale,

Yes, all those old classes are free of instances and the current class versions have a clean class history.
I now checked the class history of some of these old classes and cleaned those out as well. Funny enough, that leads to more orphans found by the script below.

From all I can find, there is only a circular reference between the class and the metaclass.

Enjoy the weekend!
Johan

On 11 Feb 2012, at 00:35, Dale Henrichs wrote:

Johan,

I asked around a bit and one of the engineers came up with a reference to an internal bug41763 (reported by commercial customer), which I've referenced from Issue 334[1].

In the body of the report it is mentioned that #findReferences: shows the GsClassDocumentation clsss while #findReferencePathToObject:, so if you haven't already tried #findReferences:, give it a shot to see what shows up ...

Dale

[1] http://code.google.com/p/glassdb/issues/detail?id=334

----- Original Message -----
| From: "Johan Brichau" <[hidden email]>
| To: "GemStone Seaside beta discussion" <[hidden email]>
| Sent: Friday, February 10, 2012 4:42:35 PM
| Subject: Re: [GS/SS Beta] instances of old class versions
|
| Hi Dale,
|
| Yes, all those old classes are free of instances and the current
| class versions have a clean class history.
| I now checked the class history of some of these old classes and
| cleaned those out as well. Funny enough, that leads to more orphans
| found by the script below.
|
| From all I can find, there is only a circular reference between the
| class and the metaclass.
|
| Enjoy the weekend!
| Johan
|
| On 11 Feb 2012, at 00:35, Dale Henrichs wrote:
|
| > Johan,
| >
| > I assume that you have already checked for instances of these
| > classes? That would be the number one reason for the classes to
| > hang around ...
| >
| > I seem to recall some "funny business" involving meta classes
| > keeping things alive, but I don't recall the details and the guys
| > that would help me remember have left for the weekend... and I
| > think that findReferencePathToObject: might also do something
| > special with classes (like possibly ignore instances), but again
| > I'm short on details.
| >
| > I also assume that you've cleaned the class histories? If there are
| > classes kept alive by a class history, they would show up as
| > orphaned and the findReferencePathToObject: might now search the
| > class history...
| >
| > If these ideas don't pan out, I'll talk to the guys no Monday and
| > see if they have any other ideas.
| >
| > Dale
| >
| > ----- Original Message -----
| > | From: "Johan Brichau" <[hidden email]>
| > | To: "GemStone Seaside beta discussion"
| > | <[hidden email]>
| > | Sent: Friday, February 10, 2012 2:52:52 PM
| > | Subject: Re: [GS/SS Beta] instances of old class versions
| > |
| > | Hi Dale,
| > |
| > | Coming back to this issue....
| > |
| > | Using your script below, I found a relatively large number of
| > | 'old'
| > | classes in my repository that only seem to be referenced by their
| > | metaclass.
| > | A search for references to find out why these classes are being
| > | hold
| > | onto always points to their metaclass and no path can be found
| > | for
| > | those metaclasses.
| > | (I'm using 'SystemRepository findReferencePathToObject: <class
| > | object>' )
| > |
| > | So, I'm puzzled why these classes remain in there. Are there any
| > | other ideas that pop into mind?
| > |
| > | thanks
| > | Johan
| > |
| > | On 24 Nov 2011, at 01:28, Dale Henrichs wrote:
| > |
| > | > Johan,
| > | >
| > | > The existence of an instance of an old class that is not in
| > | > class
| > | > history means that the instance slipped through the cracks when
| > | > the class histories were cleaned up...
| > | >
| > | > Which then implies that the instance could have been created
| > | > while
| > | > the bulk migrate was executing ...
| > | >
| > | > Which implies that it might make sense to defer the clean up of
| > | > class histories for such time when all of the vms have updated
| > | > to
| > | > a view containing the latest class version.
| > | >
| > | > Off the top of my head this could be accomplished by inserting
| > | > a
| > | > step in the bulk migrate process where a commit is performed
| > | > and
| > | > then staying in transaction until the current session is the
| > | > oldest session with the commit record...
| > | >
| > | > Have to ponder a clean way of doing this in a timely fashion,
| > | > since
| > | > it probably would involve another session doing commits to push
| > | > the commit record backlog up to the point where the stone
| > | > starts
| > | > sending out commit aborts...
| > | >
| > | > To find instances of obsolete classes I devised the following
| > | > three-step process:
| > | >
| > | >  1. Clear out the caches that might be harboring references to
| > | >  obsolete classes or otherwise keeping
| > | >     obsolete instances alive:
| > | >
| > | >     GsDeployer cleanClassHistory.
| > | >     MCDefinition clearInstances.
| > | >     MCMethodDefinition resetCachedDefinitions.
| > | >     MethodVersionHistory reset.
| > | >     MetacelloConfigurationResource reset.
| > | >     UserGlobals removeKey: #GsDeployerBar_Instances ifAbsent:
| > | >     [].
| > | >     SystemOrganizer resetSystemOrganization.
| > | >  2. Log out and log back in, then run an MFC
| > | >  3. Wait a sufficient period for dead objects to be processed
| > | >  and
| > | >  then execute the following:
| > | >      | classes orphans |
| > | >      classes := IdentitySet new.
| > | >      System myUserProfile symbolList do: [:symbolDictionary |
| > | > symbolDictionary valuesDo: [:global |
| > | > global isBehavior ifTrue: [ classes add: global ]]].
| > | >      System commitTransaction.
| > | >      orphans :=  (Metaclass allInstances collect: [:mClass |
| > | >      mClass
| > | >      theNonMetaClass ])
| > | > select: [:class | (classes includes: class) not ].
| > | >
| > | > The orphans will be a list of classes that are not reachable
| > | > from
| > | > your symbol list. The following list are system classes and can
| > | > be
| > | > safely ignored:
| > | >
| > | >   ObsoleteGeode, LargeObjectNode, NscBagLeaf, NscSetLeaf,
| > | >   NscInteriorNode, NscNode
| > | >
| > | > `GsDeployer cleanClassHistory` cleans up all of the class
| > | > histories
| > | > (reachable from your symbol list), so the remaining classes
| > | > aren't
| > | > in the class history and are being kept alive for some reason
| > | > and
| > | > are instances of those classes are candidates for migration ...
| > | >
| > | > This algorithm doesn't take into account multiple users
| > | > (SystemUser
| > | > is accounted for) nor does it take into account
| > | > GsPackageLibraries
| > | > that have not been loaded (the classes that implement
| > | > Monticello
| > | > for the base in GemStone/S 3.0.1 are an example), but this
| > | > should
| > | > be sufficient for 2.4.4.1 users. I'll update the algorithm to
| > | > be
| > | > more complete at a later date ...
| > | >
| > | > Dale
| > | >
| > | >
| > | >
| > | >
| > | > ----- Original Message -----
| > | > | From: "Johan Brichau" <[hidden email]>
| > | > | To: "GemStone Seaside beta discussion"
| > | > | <[hidden email]>
| > | > | Sent: Wednesday, November 23, 2011 10:45:44 AM
| > | > | Subject: [GS/SS Beta] instances of old class versions
| > | > |
| > | > | Hi,
| > | > |
| > | > | We encountered a strange situation today: we encountered an
| > | > | instance
| > | > | of an 'old' class in one of our repositories. This old class
| > | > | did
| > | > | not
| > | > | appear anymore in the class history of its current version
| > | > | (which
| > | > | is
| > | > | normal: I'm using GsDeployer>>doBulkMigrate) and all other
| > | > | instances
| > | > | (thousands) were correctly migrated.
| > | > |
| > | > | So, my question: is there a way to find out if there are
| > | > | still
| > | > | instances of "old" classes floating around in the system?
| > | > |
| > | > | I know that we can ask for the classhistory and check the old
| > | > | class
| > | > | versions, but this old class was no longer found in the
| > | > | classhistory
| > | > | of its current version. Just to illustrate, supposing that
| > | > | I'm
| > | > | talking about SomeClass, that means the following ('self' is
| > | > | the
| > | > | instance):
| > | > |
| > | > | self class == SomeClass
| > | > | -> false
| > | > |
| > | > | SomeClass classHistory includes: self class
| > | > | -> false
| > | > |
| > | > | SomeClass allInstances includes: self
| > | > | -> false
| > | > |
| > | > | I corrected the problem using the #migrateInstancesTo: on
| > | > | this
| > | > | instance, but now I'm left wondering if there might be others
| > | > | and
| > | > | how I could try to find those.
| > | > |
| > | > | The other question would, of course, be: "how did this
| > | > | happen"?
| > | > | I'm
| > | > | guessing that doing a live update and migrating instances
| > | > | while
| > | > | still serving Seaside requests may lead to this problem?
| > | > |
| > | > | thanks for any suggestions!
| > | > | Johan
| > |
| > |
|
|

Dale,

I confirm: GsClassDocumentation is the culprit.

We all probably need to move on to gemstone 3 ;-)

thanks!
Johan

On 13 Feb 2012, at 19:47, Dale Henrichs wrote:

I think that we can patch that son-of-a-gun ...

Dale

----- Original Message -----
| From: "Johan Brichau" <[hidden email]>
| To: "GemStone Seaside beta discussion" <[hidden email]>
| Sent: Monday, February 13, 2012 12:31:09 PM
| Subject: Re: [GS/SS Beta] instances of old class versions
|
| Dale,
|
| I confirm: GsClassDocumentation is the culprit.
|
| We all probably need to move on to gemstone 3 ;-)
|
| thanks!
| Johan
|
| On 13 Feb 2012, at 19:47, Dale Henrichs wrote:
|
| > Johan,
| >
| > I asked around a bit and one of the engineers came up with a
| > reference to an internal bug41763 (reported by commercial
| > customer), which I've referenced from Issue 334[1].
| >
| > In the body of the report it is mentioned that #findReferences:
| > shows the GsClassDocumentation clsss while
| > #findReferencePathToObject:, so if you haven't already tried
| > #findReferences:, give it a shot to see what shows up ...
| >
| > Dale
| >
| > [1] http://code.google.com/p/glassdb/issues/detail?id=334
| >
| > ----- Original Message -----
| > | From: "Johan Brichau" <[hidden email]>
| > | To: "GemStone Seaside beta discussion"
| > | <[hidden email]>
| > | Sent: Friday, February 10, 2012 4:42:35 PM
| > | Subject: Re: [GS/SS Beta] instances of old class versions
| > |
| > | Hi Dale,
| > |
| > | Yes, all those old classes are free of instances and the current
| > | class versions have a clean class history.
| > | I now checked the class history of some of these old classes and
| > | cleaned those out as well. Funny enough, that leads to more
| > | orphans
| > | found by the script below.
| > |
| > | From all I can find, there is only a circular reference between
| > | the
| > | class and the metaclass.
| > |
| > | Enjoy the weekend!
| > | Johan
| > |
| > | On 11 Feb 2012, at 00:35, Dale Henrichs wrote:
| > |
| > | > Johan,
| > | >
| > | > I assume that you have already checked for instances of these
| > | > classes? That would be the number one reason for the classes to
| > | > hang around ...
| > | >
| > | > I seem to recall some "funny business" involving meta classes
| > | > keeping things alive, but I don't recall the details and the
| > | > guys
| > | > that would help me remember have left for the weekend... and I
| > | > think that findReferencePathToObject: might also do something
| > | > special with classes (like possibly ignore instances), but
| > | > again
| > | > I'm short on details.
| > | >
| > | > I also assume that you've cleaned the class histories? If there
| > | > are
| > | > classes kept alive by a class history, they would show up as
| > | > orphaned and the findReferencePathToObject: might now search
| > | > the
| > | > class history...
| > | >
| > | > If these ideas don't pan out, I'll talk to the guys no Monday
| > | > and
| > | > see if they have any other ideas.
| > | >
| > | > Dale
| > | >
| > | > ----- Original Message -----
| > | > | From: "Johan Brichau" <[hidden email]>
| > | > | To: "GemStone Seaside beta discussion"
| > | > | <[hidden email]>
| > | > | Sent: Friday, February 10, 2012 2:52:52 PM
| > | > | Subject: Re: [GS/SS Beta] instances of old class versions
| > | > |
| > | > | Hi Dale,
| > | > |
| > | > | Coming back to this issue....
| > | > |
| > | > | Using your script below, I found a relatively large number of
| > | > | 'old'
| > | > | classes in my repository that only seem to be referenced by
| > | > | their
| > | > | metaclass.
| > | > | A search for references to find out why these classes are
| > | > | being
| > | > | hold
| > | > | onto always points to their metaclass and no path can be
| > | > | found
| > | > | for
| > | > | those metaclasses.
| > | > | (I'm using 'SystemRepository findReferencePathToObject:
| > | > | <class
| > | > | object>' )
| > | > |
| > | > | So, I'm puzzled why these classes remain in there. Are there
| > | > | any
| > | > | other ideas that pop into mind?
| > | > |
| > | > | thanks
| > | > | Johan
| > | > |
| > | > | On 24 Nov 2011, at 01:28, Dale Henrichs wrote:
| > | > |
| > | > | > Johan,
| > | > | >
| > | > | > The existence of an instance of an old class that is not in
| > | > | > class
| > | > | > history means that the instance slipped through the cracks
| > | > | > when
| > | > | > the class histories were cleaned up...
| > | > | >
| > | > | > Which then implies that the instance could have been
| > | > | > created
| > | > | > while
| > | > | > the bulk migrate was executing ...
| > | > | >
| > | > | > Which implies that it might make sense to defer the clean
| > | > | > up of
| > | > | > class histories for such time when all of the vms have
| > | > | > updated
| > | > | > to
| > | > | > a view containing the latest class version.
| > | > | >
| > | > | > Off the top of my head this could be accomplished by
| > | > | > inserting
| > | > | > a
| > | > | > step in the bulk migrate process where a commit is
| > | > | > performed
| > | > | > and
| > | > | > then staying in transaction until the current session is
| > | > | > the
| > | > | > oldest session with the commit record...
| > | > | >
| > | > | > Have to ponder a clean way of doing this in a timely
| > | > | > fashion,
| > | > | > since
| > | > | > it probably would involve another session doing commits to
| > | > | > push
| > | > | > the commit record backlog up to the point where the stone
| > | > | > starts
| > | > | > sending out commit aborts...
| > | > | >
| > | > | > To find instances of obsolete classes I devised the
| > | > | > following
| > | > | > three-step process:
| > | > | >
| > | > | >  1. Clear out the caches that might be harboring references
| > | > | >  to
| > | > | >  obsolete classes or otherwise keeping
| > | > | >     obsolete instances alive:
| > | > | >
| > | > | >     GsDeployer cleanClassHistory.
| > | > | >     MCDefinition clearInstances.
| > | > | >     MCMethodDefinition resetCachedDefinitions.
| > | > | >     MethodVersionHistory reset.
| > | > | >     MetacelloConfigurationResource reset.
| > | > | >     UserGlobals removeKey: #GsDeployerBar_Instances
| > | > | >     ifAbsent:
| > | > | >     [].
| > | > | >     SystemOrganizer resetSystemOrganization.
| > | > | >  2. Log out and log back in, then run an MFC
| > | > | >  3. Wait a sufficient period for dead objects to be
| > | > | >  processed
| > | > | >  and
| > | > | >  then execute the following:
| > | > | >      | classes orphans |
| > | > | >      classes := IdentitySet new.
| > | > | >      System myUserProfile symbolList do: [:symbolDictionary
| > | > | >      |
| > | > | > symbolDictionary valuesDo: [:global |
| > | > | > global isBehavior ifTrue: [ classes add: global ]]].
| > | > | >      System commitTransaction.
| > | > | >      orphans :=  (Metaclass allInstances collect: [:mClass
| > | > | >      |
| > | > | >      mClass
| > | > | >      theNonMetaClass ])
| > | > | > select: [:class | (classes includes: class) not ].
| > | > | >
| > | > | > The orphans will be a list of classes that are not
| > | > | > reachable
| > | > | > from
| > | > | > your symbol list. The following list are system classes and
| > | > | > can
| > | > | > be
| > | > | > safely ignored:
| > | > | >
| > | > | >   ObsoleteGeode, LargeObjectNode, NscBagLeaf, NscSetLeaf,
| > | > | >   NscInteriorNode, NscNode
| > | > | >
| > | > | > `GsDeployer cleanClassHistory` cleans up all of the class
| > | > | > histories
| > | > | > (reachable from your symbol list), so the remaining classes
| > | > | > aren't
| > | > | > in the class history and are being kept alive for some
| > | > | > reason
| > | > | > and
| > | > | > are instances of those classes are candidates for migration
| > | > | > ...
| > | > | >
| > | > | > This algorithm doesn't take into account multiple users
| > | > | > (SystemUser
| > | > | > is accounted for) nor does it take into account
| > | > | > GsPackageLibraries
| > | > | > that have not been loaded (the classes that implement
| > | > | > Monticello
| > | > | > for the base in GemStone/S 3.0.1 are an example), but this
| > | > | > should
| > | > | > be sufficient for 2.4.4.1 users. I'll update the algorithm
| > | > | > to
| > | > | > be
| > | > | > more complete at a later date ...
| > | > | >
| > | > | > Dale
| > | > | >
| > | > | >
| > | > | >
| > | > | >
| > | > | > ----- Original Message -----
| > | > | > | From: "Johan Brichau" <[hidden email]>
| > | > | > | To: "GemStone Seaside beta discussion"
| > | > | > | <[hidden email]>
| > | > | > | Sent: Wednesday, November 23, 2011 10:45:44 AM
| > | > | > | Subject: [GS/SS Beta] instances of old class versions
| > | > | > |
| > | > | > | Hi,
| > | > | > |
| > | > | > | We encountered a strange situation today: we encountered
| > | > | > | an
| > | > | > | instance
| > | > | > | of an 'old' class in one of our repositories. This old
| > | > | > | class
| > | > | > | did
| > | > | > | not
| > | > | > | appear anymore in the class history of its current
| > | > | > | version
| > | > | > | (which
| > | > | > | is
| > | > | > | normal: I'm using GsDeployer>>doBulkMigrate) and all
| > | > | > | other
| > | > | > | instances
| > | > | > | (thousands) were correctly migrated.
| > | > | > |
| > | > | > | So, my question: is there a way to find out if there are
| > | > | > | still
| > | > | > | instances of "old" classes floating around in the system?
| > | > | > |
| > | > | > | I know that we can ask for the classhistory and check the
| > | > | > | old
| > | > | > | class
| > | > | > | versions, but this old class was no longer found in the
| > | > | > | classhistory
| > | > | > | of its current version. Just to illustrate, supposing
| > | > | > | that
| > | > | > | I'm
| > | > | > | talking about SomeClass, that means the following ('self'
| > | > | > | is
| > | > | > | the
| > | > | > | instance):
| > | > | > |
| > | > | > | self class == SomeClass
| > | > | > | -> false
| > | > | > |
| > | > | > | SomeClass classHistory includes: self class
| > | > | > | -> false
| > | > | > |
| > | > | > | SomeClass allInstances includes: self
| > | > | > | -> false
| > | > | > |
| > | > | > | I corrected the problem using the #migrateInstancesTo: on
| > | > | > | this
| > | > | > | instance, but now I'm left wondering if there might be
| > | > | > | others
| > | > | > | and
| > | > | > | how I could try to find those.
| > | > | > |
| > | > | > | The other question would, of course, be: "how did this
| > | > | > | happen"?
| > | > | > | I'm
| > | > | > | guessing that doing a live update and migrating instances
| > | > | > | while
| > | > | > | still serving Seaside requests may lead to this problem?
| > | > | > |
| > | > | > | thanks for any suggestions!
| > | > | > | Johan
| > | > |
| > | > |
| > |
| > |
|
|

Johan,

It looks like I included the fix for Issue 334 in GLASS 1.0-beta.8.7.1 .... but having the fix doesn't fix classes that haven't been updated since upgrading to
1.0-beta.8.7.1.

I haven't tested the script, but something like the following should patch things (GsClassDocumentation>>itsClass: from GLASS 1.0-beta.8.7.1 is needed...):

     System myUserProfile symbolList do: [:symbolDictionary |
       symbolDictionary valuesDo: [:global |
         global isBehavior ifTrue: [ | desc |
           (desc := global description) notNil
             ifTrue: [
               description itsClass ~~ global
                 ifTrue: [ global description itsClass: global ]]]].

Be wary that there might be permission issues if this condition exists in a class in Globals (in which case we'd like to know).

Dale

----- Original Message -----
| From: "Dale Henrichs" <[hidden email]>
| To: "GemStone Seaside beta discussion" <[hidden email]>
| Sent: Monday, February 13, 2012 1:01:48 PM
| Subject: Re: [GS/SS Beta] instances of old class versions
|
| I think that we can patch that son-of-a-gun ...
|
| Dale
|
| ----- Original Message -----
| | From: "Johan Brichau" <[hidden email]>
| | To: "GemStone Seaside beta discussion" <[hidden email]>
| | Sent: Monday, February 13, 2012 12:31:09 PM
| | Subject: Re: [GS/SS Beta] instances of old class versions
| |
| | Dale,
| |
| | I confirm: GsClassDocumentation is the culprit.
| |
| | We all probably need to move on to gemstone 3 ;-)
| |
| | thanks!
| | Johan
| |
| | On 13 Feb 2012, at 19:47, Dale Henrichs wrote:
| |
| | > Johan,
| | >
| | > I asked around a bit and one of the engineers came up with a
| | > reference to an internal bug41763 (reported by commercial
| | > customer), which I've referenced from Issue 334[1].
| | >
| | > In the body of the report it is mentioned that #findReferences:
| | > shows the GsClassDocumentation clsss while
| | > #findReferencePathToObject:, so if you haven't already tried
| | > #findReferences:, give it a shot to see what shows up ...
| | >
| | > Dale
| | >
| | > [1] http://code.google.com/p/glassdb/issues/detail?id=334
| | >
| | > ----- Original Message -----
| | > | From: "Johan Brichau" <[hidden email]>
| | > | To: "GemStone Seaside beta discussion"
| | > | <[hidden email]>
| | > | Sent: Friday, February 10, 2012 4:42:35 PM
| | > | Subject: Re: [GS/SS Beta] instances of old class versions
| | > |
| | > | Hi Dale,
| | > |
| | > | Yes, all those old classes are free of instances and the
| | > | current
| | > | class versions have a clean class history.
| | > | I now checked the class history of some of these old classes
| | > | and
| | > | cleaned those out as well. Funny enough, that leads to more
| | > | orphans
| | > | found by the script below.
| | > |
| | > | From all I can find, there is only a circular reference between
| | > | the
| | > | class and the metaclass.
| | > |
| | > | Enjoy the weekend!
| | > | Johan
| | > |
| | > | On 11 Feb 2012, at 00:35, Dale Henrichs wrote:
| | > |
| | > | > Johan,
| | > | >
| | > | > I assume that you have already checked for instances of these
| | > | > classes? That would be the number one reason for the classes
| | > | > to
| | > | > hang around ...
| | > | >
| | > | > I seem to recall some "funny business" involving meta classes
| | > | > keeping things alive, but I don't recall the details and the
| | > | > guys
| | > | > that would help me remember have left for the weekend... and
| | > | > I
| | > | > think that findReferencePathToObject: might also do something
| | > | > special with classes (like possibly ignore instances), but
| | > | > again
| | > | > I'm short on details.
| | > | >
| | > | > I also assume that you've cleaned the class histories? If
| | > | > there
| | > | > are
| | > | > classes kept alive by a class history, they would show up as
| | > | > orphaned and the findReferencePathToObject: might now search
| | > | > the
| | > | > class history...
| | > | >
| | > | > If these ideas don't pan out, I'll talk to the guys no Monday
| | > | > and
| | > | > see if they have any other ideas.
| | > | >
| | > | > Dale
| | > | >
| | > | > ----- Original Message -----
| | > | > | From: "Johan Brichau" <[hidden email]>
| | > | > | To: "GemStone Seaside beta discussion"
| | > | > | <[hidden email]>
| | > | > | Sent: Friday, February 10, 2012 2:52:52 PM
| | > | > | Subject: Re: [GS/SS Beta] instances of old class versions
| | > | > |
| | > | > | Hi Dale,
| | > | > |
| | > | > | Coming back to this issue....
| | > | > |
| | > | > | Using your script below, I found a relatively large number
| | > | > | of
| | > | > | 'old'
| | > | > | classes in my repository that only seem to be referenced by
| | > | > | their
| | > | > | metaclass.
| | > | > | A search for references to find out why these classes are
| | > | > | being
| | > | > | hold
| | > | > | onto always points to their metaclass and no path can be
| | > | > | found
| | > | > | for
| | > | > | those metaclasses.
| | > | > | (I'm using 'SystemRepository findReferencePathToObject:
| | > | > | <class
| | > | > | object>' )
| | > | > |
| | > | > | So, I'm puzzled why these classes remain in there. Are
| | > | > | there
| | > | > | any
| | > | > | other ideas that pop into mind?
| | > | > |
| | > | > | thanks
| | > | > | Johan
| | > | > |
| | > | > | On 24 Nov 2011, at 01:28, Dale Henrichs wrote:
| | > | > |
| | > | > | > Johan,
| | > | > | >
| | > | > | > The existence of an instance of an old class that is not
| | > | > | > in
| | > | > | > class
| | > | > | > history means that the instance slipped through the
| | > | > | > cracks
| | > | > | > when
| | > | > | > the class histories were cleaned up...
| | > | > | >
| | > | > | > Which then implies that the instance could have been
| | > | > | > created
| | > | > | > while
| | > | > | > the bulk migrate was executing ...
| | > | > | >
| | > | > | > Which implies that it might make sense to defer the clean
| | > | > | > up of
| | > | > | > class histories for such time when all of the vms have
| | > | > | > updated
| | > | > | > to
| | > | > | > a view containing the latest class version.
| | > | > | >
| | > | > | > Off the top of my head this could be accomplished by
| | > | > | > inserting
| | > | > | > a
| | > | > | > step in the bulk migrate process where a commit is
| | > | > | > performed
| | > | > | > and
| | > | > | > then staying in transaction until the current session is
| | > | > | > the
| | > | > | > oldest session with the commit record...
| | > | > | >
| | > | > | > Have to ponder a clean way of doing this in a timely
| | > | > | > fashion,
| | > | > | > since
| | > | > | > it probably would involve another session doing commits
| | > | > | > to
| | > | > | > push
| | > | > | > the commit record backlog up to the point where the stone
| | > | > | > starts
| | > | > | > sending out commit aborts...
| | > | > | >
| | > | > | > To find instances of obsolete classes I devised the
| | > | > | > following
| | > | > | > three-step process:
| | > | > | >
| | > | > | >  1. Clear out the caches that might be harboring
| | > | > | >  references
| | > | > | >  to
| | > | > | >  obsolete classes or otherwise keeping
| | > | > | >     obsolete instances alive:
| | > | > | >
| | > | > | >     GsDeployer cleanClassHistory.
| | > | > | >     MCDefinition clearInstances.
| | > | > | >     MCMethodDefinition resetCachedDefinitions.
| | > | > | >     MethodVersionHistory reset.
| | > | > | >     MetacelloConfigurationResource reset.
| | > | > | >     UserGlobals removeKey: #GsDeployerBar_Instances
| | > | > | >     ifAbsent:
| | > | > | >     [].
| | > | > | >     SystemOrganizer resetSystemOrganization.
| | > | > | >  2. Log out and log back in, then run an MFC
| | > | > | >  3. Wait a sufficient period for dead objects to be
| | > | > | >  processed
| | > | > | >  and
| | > | > | >  then execute the following:
| | > | > | >      | classes orphans |
| | > | > | >      classes := IdentitySet new.
| | > | > | >      System myUserProfile symbolList do:
| | > | > | >      [:symbolDictionary
| | > | > | >      |
| | > | > | > symbolDictionary valuesDo: [:global |
| | > | > | > global isBehavior ifTrue: [ classes add: global ]]].
| | > | > | >      System commitTransaction.
| | > | > | >      orphans :=  (Metaclass allInstances collect:
| | > | > | >      [:mClass
| | > | > | >      |
| | > | > | >      mClass
| | > | > | >      theNonMetaClass ])
| | > | > | > select: [:class | (classes includes: class) not ].
| | > | > | >
| | > | > | > The orphans will be a list of classes that are not
| | > | > | > reachable
| | > | > | > from
| | > | > | > your symbol list. The following list are system classes
| | > | > | > and
| | > | > | > can
| | > | > | > be
| | > | > | > safely ignored:
| | > | > | >
| | > | > | >   ObsoleteGeode, LargeObjectNode, NscBagLeaf, NscSetLeaf,
| | > | > | >   NscInteriorNode, NscNode
| | > | > | >
| | > | > | > `GsDeployer cleanClassHistory` cleans up all of the class
| | > | > | > histories
| | > | > | > (reachable from your symbol list), so the remaining
| | > | > | > classes
| | > | > | > aren't
| | > | > | > in the class history and are being kept alive for some
| | > | > | > reason
| | > | > | > and
| | > | > | > are instances of those classes are candidates for
| | > | > | > migration
| | > | > | > ...
| | > | > | >
| | > | > | > This algorithm doesn't take into account multiple users
| | > | > | > (SystemUser
| | > | > | > is accounted for) nor does it take into account
| | > | > | > GsPackageLibraries
| | > | > | > that have not been loaded (the classes that implement
| | > | > | > Monticello
| | > | > | > for the base in GemStone/S 3.0.1 are an example), but
| | > | > | > this
| | > | > | > should
| | > | > | > be sufficient for 2.4.4.1 users. I'll update the
| | > | > | > algorithm
| | > | > | > to
| | > | > | > be
| | > | > | > more complete at a later date ...
| | > | > | >
| | > | > | > Dale
| | > | > | >
| | > | > | >
| | > | > | >
| | > | > | >
| | > | > | > ----- Original Message -----
| | > | > | > | From: "Johan Brichau" <[hidden email]>
| | > | > | > | To: "GemStone Seaside beta discussion"
| | > | > | > | <[hidden email]>
| | > | > | > | Sent: Wednesday, November 23, 2011 10:45:44 AM
| | > | > | > | Subject: [GS/SS Beta] instances of old class versions
| | > | > | > |
| | > | > | > | Hi,
| | > | > | > |
| | > | > | > | We encountered a strange situation today: we
| | > | > | > | encountered
| | > | > | > | an
| | > | > | > | instance
| | > | > | > | of an 'old' class in one of our repositories. This old
| | > | > | > | class
| | > | > | > | did
| | > | > | > | not
| | > | > | > | appear anymore in the class history of its current
| | > | > | > | version
| | > | > | > | (which
| | > | > | > | is
| | > | > | > | normal: I'm using GsDeployer>>doBulkMigrate) and all
| | > | > | > | other
| | > | > | > | instances
| | > | > | > | (thousands) were correctly migrated.
| | > | > | > |
| | > | > | > | So, my question: is there a way to find out if there
| | > | > | > | are
| | > | > | > | still
| | > | > | > | instances of "old" classes floating around in the
| | > | > | > | system?
| | > | > | > |
| | > | > | > | I know that we can ask for the classhistory and check
| | > | > | > | the
| | > | > | > | old
| | > | > | > | class
| | > | > | > | versions, but this old class was no longer found in the
| | > | > | > | classhistory
| | > | > | > | of its current version. Just to illustrate, supposing
| | > | > | > | that
| | > | > | > | I'm
| | > | > | > | talking about SomeClass, that means the following
| | > | > | > | ('self'
| | > | > | > | is
| | > | > | > | the
| | > | > | > | instance):
| | > | > | > |
| | > | > | > | self class == SomeClass
| | > | > | > | -> false
| | > | > | > |
| | > | > | > | SomeClass classHistory includes: self class
| | > | > | > | -> false
| | > | > | > |
| | > | > | > | SomeClass allInstances includes: self
| | > | > | > | -> false
| | > | > | > |
| | > | > | > | I corrected the problem using the #migrateInstancesTo:
| | > | > | > | on
| | > | > | > | this
| | > | > | > | instance, but now I'm left wondering if there might be
| | > | > | > | others
| | > | > | > | and
| | > | > | > | how I could try to find those.
| | > | > | > |
| | > | > | > | The other question would, of course, be: "how did this
| | > | > | > | happen"?
| | > | > | > | I'm
| | > | > | > | guessing that doing a live update and migrating
| | > | > | > | instances
| | > | > | > | while
| | > | > | > | still serving Seaside requests may lead to this
| | > | > | > | problem?
| | > | > | > |
| | > | > | > | thanks for any suggestions!
| | > | > | > | Johan
| | > | > |
| | > | > |
| | > |
| | > |
| |
| |
|