Migrating from 3.2.9 to 3.3.3 block compilation issue

Mariano,

I'll dig into this later this afternoon (I'm out of time right now) ... All of the code is reloaded at each upgrade, so it is likely that there are few blocks that are hanging around (SortedCollections, etc.) that need to be rebuilt ...

Hi guys, 

I am migrating some stones (tODE and GsDevKit based) that are on GemStone 3.2.9 to 3.3.3. I am doing this with the `upgradeStone` gsDevKit command. 

I first hit one problem when trying to execute this code via topaz:

MCRepositoryGroup default removeRepository: (MCHttpRepository

                        location: 'xxx'

                        user: ''

                        password: '').

ERROR 2142 , a ImproperOperation occurred (error 2142), Cannot execute method, 'method needs recompile, MCRepositoryGroup class >> default , oop 206736385' (ImproperOperation)

Then, I said...ok, let's connect with tODE and investigate some more (like doing Object _objectForOop: 206736385). But when I try to connect with tODE I get:

Error running testLogin:

'Error: Unable to log into server:a ImproperOperation occurred (error 2142), Cannot execute method, ''method needs recompile, TDTopezServer class >> createFor: , oop 245949953''with: ''TDTopezServer createFor: 1'''

NOTE: netldi log can be found in the file: '$GS_HOME/server/stones/<stone-name>/logs/netldi.log'.

So...looks like I have methods to be recompiled everywhere. I saw this issue [1] but that does not seem to help as these closures are not in sorted collections right ?

Now, manually fixing each, like this:

 | sc sortBlock val cls str val |
  sc := "sortedcollection instance".
  sortBlock := sc sortBlock.
  str := sortBlock _sourceString.
  val := str evaluate.
  sc _sortBlock: val

Is a pain...because I must know how to re-assign the new compiled method. Is there an alternative GENERIC solution to recompile all needed ones? I can think of that but using #become:

System commit. 

  [ 

  MCPlatformSupport installAlmostOutOfMemoryStaticHandler: 75.

  ExecBlock allSubInstances do: [:each | 

each become: each _sourceString evaluate. 

].

]

  ensure: [ MCPlatformSupport uninstallAlmostOutOfMemoryStaticHandler ].

System commit. 

But...comment of #become: says that BlockClosure instances cannot be become...BUUUUUUH. 

So.. any idea?

Thanks in advance,

[1] https://github.com/GsDevKit/GsDevKit_home/issues/70

--

Mariano
http://marianopeck.wordpress.com

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

Mariano,

After looking at your log files, it looks like this is a bug in upgradeStone script that is specific to the gs_port branch --- related to using a non-DataCurator user for GsDevKit --- I've updated the gs_port branch with my proposed fix ...

Dale

Dale, after checking the logs in upgradeLog directory, I found out that I had errors in file `upgradeTo3x.out ` pasted below.

BTW, I can share with you a zip with all files if you want to. 

Cheers,

topaz 1>

topaz 1> ! clear caches - those that are unneeded or create problem during initial load

topaz 1> run

"MCDefinition clearInstances"

MCDefinition _classVars at: #Instances put: nil.

"MCMethodDefinition shutdown"

MCMethodDefinition _classVars at: #Definitions put: nil.

"MethodVersionHistory reset"

UserGlobals removeKey: #MethodVersionHistory_UniqueInstance ifAbsent: [].

"clear out PackageOrganizer default class instance variable"

PackageOrganizer atClassInstVar: #default put: nil.

"clear out McCacheRepository default class instance variable"

MCCacheRepository atClassInstVar: #default put: nil.

"wipe out the working copy registry ... will be redefined later"

MCPackageManager atClassInstVar: #registry put: nil.

MCWorkingCopy atClassInstVar: #registry put: nil.

"wipe out the default MCRepositoryGroup"

MCRepositoryGroup atClassInstVar: #default put: nil.

"Wipe out the Metacello registry ... if it exists"

(GsCurrentSession currentSession symbolList objectNamed: #MetacelloProjectRegistration)

  ifNotNil: [:cl | cl _classVars at: #Registry put: nil ].

true

%

GemStone Smalltalk Compiler Errors:

   "MCDefinition clearInstances"

   MCDefinition _classVars at: #Instances put: nil.

 * ^1                                                                 *******

   "MCMethodDefinition shutdown"

   MCMethodDefinition _classVars at: #Definitions put: nil.

 * ^2                                                                 *******

   "MethodVersionHistory reset"

   UserGlobals removeKey: #MethodVersionHistory_UniqueInstance ifAbsent: [].

   "clear out PackageOrganizer default class instance variable"

   PackageOrganizer atClassInstVar: #default put: nil.

 * ^3                                                                 *******

   "clear out McCacheRepository default class instance variable"

   MCCacheRepository atClassInstVar: #default put: nil.

 * ^4                                                                 *******

   "wipe out the working copy registry ... will be redefined later"

   MCPackageManager atClassInstVar: #registry put: nil.

 * ^5                                                                 *******

   MCWorkingCopy atClassInstVar: #registry put: nil.

 * ^6                                                                 *******

   "wipe out the default MCRepositoryGroup"

   MCRepositoryGroup atClassInstVar: #default put: nil.

 * ^7                                                                 *******

   "Wipe out the Metacello registry ... if it exists"

   (GsCurrentSession currentSession symbolList objectNamed: #MetacelloProjectRegistration)

     ifNotNil: [:cl | cl _classVars at: #Registry put: nil ].

   true

1: [1031] undefined symbol

2: [1031] undefined symbol

3: [1031] undefined symbol

4: [1031] undefined symbol

5: [1031] undefined symbol

6: [1031] undefined symbol

7: [1031] undefined symbol

ERROR: UNEXPECTED ERROR

topaz> time

 01/26/2017 11:50:27.993 EST

topaz > exec iferr 1 : stk

Stack is not active

topaz > exec iferr 2 : stack

Stack is not active

topaz > exec iferr 3 : input pop

topaz > exec iferr 4 : exit

Logging out session 1.

On Thu, Jan 26, 2017 at 6:11 PM, Dale Henrichs via Glass <[hidden email]> wrote:

Mariano,

I'll dig into this later this afternoon (I'm out of time right now) ... All of the code is reloaded at each upgrade, so it is likely that there are few blocks that are hanging around (SortedCollections, etc.) that need to be rebuilt ...

Dale

On 01/26/2017 11:01 AM, Mariano Martinez Peck via Glass wrote:

Hi guys, 

I am migrating some stones (tODE and GsDevKit based) that are on GemStone 3.2.9 to 3.3.3. I am doing this with the `upgradeStone` gsDevKit command. 

I first hit one problem when trying to execute this code via topaz:

MCRepositoryGroup default removeRepository: (MCHttpRepository

                        location: 'xxx'

                        user: ''

                        password: '').

ERROR 2142 , a ImproperOperation occurred (error 2142), Cannot execute method, 'method needs recompile, MCRepositoryGroup class >> default , oop 206736385' (ImproperOperation)

Then, I said...ok, let's connect with tODE and investigate some more (like doing Object _objectForOop: 206736385). But when I try to connect with tODE I get:

Error running testLogin:

'Error: Unable to log into server:a ImproperOperation occurred (error 2142), Cannot execute method, ''method needs recompile, TDTopezServer class >> createFor: , oop 245949953''with: ''TDTopezServer createFor: 1'''

NOTE: netldi log can be found in the file: '$GS_HOME/server/stones/<stone-name>/logs/netldi.log'.

So...looks like I have methods to be recompiled everywhere. I saw this issue [1] but that does not seem to help as these closures are not in sorted collections right ?

Now, manually fixing each, like this:

 | sc sortBlock val cls str val |
  sc := "sortedcollection instance".
  sortBlock := sc sortBlock.
  str := sortBlock _sourceString.
  val := str evaluate.
  sc _sortBlock: val

Is a pain...because I must know how to re-assign the new compiled method. Is there an alternative GENERIC solution to recompile all needed ones? I can think of that but using #become:

System commit. 

  [ 

  MCPlatformSupport installAlmostOutOfMemoryStaticHandler: 75.

  ExecBlock allSubInstances do: [:each | 

each become: each _sourceString evaluate. 

].

]

  ensure: [ MCPlatformSupport uninstallAlmostOutOfMemoryStaticHandler ].

System commit. 

But...comment of #become: says that BlockClosure instances cannot be become...BUUUUUUH. 

So.. any idea?

Thanks in advance,

[1] https://github.com/GsDevKit/GsDevKit_home/issues/70

--

Mariano
http://marianopeck.wordpress.com

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

_______________________________________________ Glass mailing list [hidden email] http://lists.gemtalksystems.com/mailman/listinfo/glass

--

Mariano http://marianopeck.wordpress.com

Mariano,

1. Find allInstances of ExecBlock

2. Select all instances that #needsRecompile.

3. Send #recompile to the selected instances and collect instances that signal an error during #recompile. The #recompile method attempts to recompile Simple blocks in-place if there is an error during compilation or the block is not simple, an Error is signaled.

4. For the remaining instances of ExecBlock, you will have to track down the reference to the block and reinstall a new, recompiled instance of the ExecBlock.

    1. look at the #blockSource and the #receiver for the block and you might recognize the method needed to recreate the block

    2. look at the #blockSource and pick out a likely chunk of source that you can use to search in method source (`browse source --exact `<source-chunk>`)

    3. Use Repository>>findReferencePathToObject: or Repository>>findAllReferencePathsToObject: to find which object(s) are holding onto the block.

Dale

Hi guys,

Ok, as Dale found out, some automatic recompilation from `upgradeStone` was not being done because of a GsDevKit_issue.

Now, problem is that even with that fixed, I am still finding here and there more and more closures that need recompilation. These are NOT sort blocks of sorted collections nor method. They are closures (from my domain objects) stored somewhere. 

Now, manually fixing each, like this:

anObject setter: anObject getter _sourceString evaluate.

Is a pain and not accurate (the closure may be still referenced from another place)...because I must know how to re-assign the new closure. Is there an alternative GENERIC solution to recompile all needed ones? I can think of that but using #become:

System commit. 

  [ 

  MCPlatformSupport installAlmostOutOfMemoryStaticHandler: 75.

    ExecBlock allSubInstances do: [:each | 

each become: each _sourceString evaluate. 

].

]

  ensure: [ MCPlatformSupport uninstallAlmostOutOfMemoryStaticHandler ].

System commit. 

But...comment of #become: says that BlockClosure instances cannot be become...BUUUUUUH. 

So.. any idea?

Thanks in advance,

On Thu, Jan 26, 2017 at 4:01 PM, Mariano Martinez Peck <[hidden email]> wrote:

Hi guys, 

I am migrating some stones (tODE and GsDevKit based) that are on GemStone 3.2.9 to 3.3.3. I am doing this with the `upgradeStone` gsDevKit command. 

I first hit one problem when trying to execute this code via topaz:

MCRepositoryGroup default removeRepository: (MCHttpRepository

                        location: 'xxx'

                        user: ''

                        password: '').

ERROR 2142 , a ImproperOperation occurred (error 2142), Cannot execute method, 'method needs recompile, MCRepositoryGroup class >> default , oop 206736385' (ImproperOperation)

Then, I said...ok, let's connect with tODE and investigate some more (like doing Object _objectForOop: 206736385). But when I try to connect with tODE I get:

Error running testLogin:

'Error: Unable to log into server:a ImproperOperation occurred (error 2142), Cannot execute method, ''method needs recompile, TDTopezServer class >> createFor: , oop 245949953''with: ''TDTopezServer createFor: 1'''

NOTE: netldi log can be found in the file: '$GS_HOME/server/stones/<stone-name>/logs/netldi.log'.

So...looks like I have methods to be recompiled everywhere. I saw this issue [1] but that does not seem to help as these closures are not in sorted collections right ?

Now, manually fixing each, like this:

 | sc sortBlock val cls str val |
  sc := "sortedcollection instance".
  sortBlock := sc sortBlock.
  str := sortBlock _sourceString.
  val := str evaluate.
  sc _sortBlock: val

Is a pain...because I must know how to re-assign the new compiled method. Is there an alternative GENERIC solution to recompile all needed ones? I can think of that but using #become:

System commit. 

  [ 

  MCPlatformSupport installAlmostOutOfMemoryStaticHandler: 75.

  ExecBlock allSubInstances do: [:each | 

each become: each _sourceString evaluate. 

].

]

  ensure: [ MCPlatformSupport uninstallAlmostOutOfMemoryStaticHandler ].

System commit. 

But...comment of #become: says that BlockClosure instances cannot be become...BUUUUUUH. 

So.. any idea?

Thanks in advance,

[1] https://github.com/GsDevKit/GsDevKit_home/issues/70

--

Mariano
http://marianopeck.wordpress.com

--

Mariano
http://marianopeck.wordpress.com

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

On Tue, Jan 31, 2017 at 2:25 PM, Dale Henrichs via Glass <[hidden email]> wrote:

Mariano,

1. Find allInstances of ExecBlock

2. Select all instances that #needsRecompile.

3. Send #recompile to the selected instances and collect instances that signal an error during #recompile. The #recompile method attempts to recompile Simple blocks in-place if there is an error during compilation or the block is not simple, an Error is signaled.

4. For the remaining instances of ExecBlock, you will have to track down the reference to the block and reinstall a new, recompiled instance of the ExecBlock.

    1. look at the #blockSource and the #receiver for the block and you might recognize the method needed to recreate the block

    2. look at the #blockSource and pick out a likely chunk of source that you can use to search in method source (`browse source --exact `<source-chunk>`)

    3. Use Repository>>findReferencePathToObject: or Repository>>findAllReferencePathsToObject: to find which object(s) are holding onto the block.

Hi Dale,

OK, it seems I am not having good luck. I tried the following code:

[

| allBlocks allBlocksNeedingRecompile blocksFailedToRecompile correctlyRecompiled | 

blocksFailedToRecompile := OrderedCollection new. 

correctlyRecompiled := OrderedCollection new. 

System commit. 

allBlocks := ExecBlock allSubInstances.

allBlocksNeedingRecompile := allBlocks select: [:each | each needsRecompile ]. 

allBlocksNeedingRecompile do: [:each |

[ 

each recompile.

correctlyRecompiled add: each.

  ] on: Error do: [:ex | 

blocksFailedToRecompile add: each.

]

].

System commit. 

Transcript show:  'Correctly recompiled closures: ',  correctlyRecompiled size asString.

Transcript show: 'Erred recompiled closures: ',  blocksFailedToRecompile size asString.

] on: Warning do: [:ex | ex resume  ]

And it outputs:

Correctly recompiled closures: 0Erred recompiled closures: 797155

Wow. That's a lot to fix. 

But things get strange when I analyze which are the failing closures. For example,  I have an ExecBlock whose source is:

"This is source for a block.  " 

 ^ [ 

     self error:

         'No concrete db roles associated to '''

           , (FaDBAccessor common users row: username column: 'CONCRETE ROLE')

           , ''' of user ''' , username

           ,

             '''. Either the concrete db role id is wrong or the roles must be added to concreteRoles.csv' 

]

That closure, is a closure of an ifEmpty:  of the method #sessionMapFor:prepareAccessLimitingSessionsWith:

See:

sessionMapFor: username prepareAccessLimitingSessionsWith: aDictionaryOfPreparationBlocks

| answer  concreteRole abstractRoleID  |

answer := IdentityDictionary new.

concreteRole := self concreteRoleForUsername: username. 

concreteRole

   ifEmpty: [ 

     self error:

         'No concrete db roles associated to '''

           , (FaDBAccessor common users row: username column: 'CONCRETE ROLE')

           , ''' of user ''' , username

           ,

             '''. Either the concrete db role id is wrong or the roles must be added to concreteRoles.csv' 

].

.....

What is going on? I re-load all my app code after I upgraded stone so I expect that method to have been recompiled. In fact, I just edited the method and saved it again (so that a recompilation was triggered) and the above closure still appears as needing compilation!

Even worst....   the _method  instVar of that example IS NOT IDENTICAL (same OOP) to the one installed in the method dictionary (FaDBSessionsByRolesFactory >> #sessionMapFor:prepareAccessLimitingSessionsWith: ). So it looks like if they were un-installed closures. 

I thought it could be some MC version history around or whatever. So I performed my typical GC and cleanup code which should collect all that garbage. Still same numbers.

mmmm I am running out of ideas. 

Thoughts?

On Wed, Feb 1, 2017 at 2:20 AM, Dale Henrichs <[hidden email]> wrote:

On 1/31/17 7:44 PM, Mariano Martinez Peck wrote:

On Tue, Jan 31, 2017 at 2:25 PM, Dale Henrichs via Glass <[hidden email]> wrote:

Mariano,

1. Find allInstances of ExecBlock

2. Select all instances that #needsRecompile.

3. Send #recompile to the selected instances and collect instances that signal an error during #recompile. The #recompile method attempts to recompile Simple blocks in-place if there is an error during compilation or the block is not simple, an Error is signaled.

4. For the remaining instances of ExecBlock, you will have to track down the reference to the block and reinstall a new, recompiled instance of the ExecBlock.

    1. look at the #blockSource and the #receiver for the block and you might recognize the method needed to recreate the block

    2. look at the #blockSource and pick out a likely chunk of source that you can use to search in method source (`browse source --exact `<source-chunk>`)

    3. Use Repository>>findReferencePathToObject: or Repository>>findAllReferencePathsToObject: to find which object(s) are holding onto the block.

Hi Dale,

OK, it seems I am not having good luck. I tried the following code:

[

| allBlocks allBlocksNeedingRecompile blocksFailedToRecompile correctlyRecompiled | 

blocksFailedToRecompile := OrderedCollection new. 

correctlyRecompiled := OrderedCollection new. 

System commit. 

allBlocks := ExecBlock allSubInstances.

allBlocksNeedingRecompile := allBlocks select: [:each | each needsRecompile ]. 

allBlocksNeedingRecompile do: [:each |

[ 

each recompile.

correctlyRecompiled add: each.

  ] on: Error do: [:ex | 

blocksFailedToRecompile add: each.

]

].

System commit. 

Transcript show:  'Correctly recompiled closures: ',  correctlyRecompiled size asString.

Transcript show: 'Erred recompiled closures: ',  blocksFailedToRecompile size asString.

] on: Warning do: [:ex | ex resume  ]

And it outputs:

Correctly recompiled closures: 0Erred recompiled closures: 797155

Wow. That's a lot to fix. 

But things get strange when I analyze which are the failing closures. For example,  I have an ExecBlock whose source is:

"This is source for a block.  " 

 ^ [ 

     self error:

         'No concrete db roles associated to '''

           , (FaDBAccessor common users row: username column: 'CONCRETE ROLE')

           , ''' of user ''' , username

           ,

             '''. Either the concrete db role id is wrong or the roles must be added to concreteRoles.csv' 

]

That closure, is a closure of an ifEmpty:  of the method #sessionMapFor:prepareAccessLimitingSessionsWith:

See:

sessionMapFor: username prepareAccessLimitingSessionsWith: aDictionaryOfPreparationBlocks

| answer  concreteRole abstractRoleID  |

answer := IdentityDictionary new.

concreteRole := self concreteRoleForUsername: username. 

concreteRole

   ifEmpty: [ 

     self error:

         'No concrete db roles associated to '''

           , (FaDBAccessor common users row: username column: 'CONCRETE ROLE')

           , ''' of user ''' , username

           ,

             '''. Either the concrete db role id is wrong or the roles must be added to concreteRoles.csv' 

].

.....

What is going on? I re-load all my app code after I upgraded stone so I expect that method to have been recompiled. In fact, I just edited the method and saved it again (so that a recompilation was triggered) and the above closure still appears as needing compilation!

Even worst....   the _method  instVar of that example IS NOT IDENTICAL (same OOP) to the one installed in the method dictionary (FaDBSessionsByRolesFactory >> #sessionMapFor:prepareAccessLimitingSessionsWith: ). So it looks like if they were un-installed closures. 

I thought it could be some MC version history around or whatever. So I performed my typical GC and cleanup code which should collect all that garbage. Still same numbers.

mmmm I am running out of ideas. 

Thoughts?

I imagine that the #sessionMapFor:prepareAccessLimitingSessionsWith: method is in an older version of the class, which would explain the differences in oops and the fact that when you recompile the method in the current version of the class, you don't eliminate block that needs to be recompiled ...

`GsDeployer new classesWithClassHistory` should give you the list of classes that have older versions ...

Hi Dale,

I tried with that and it answers an empty array. 

 

Now, I tried in a much smaller stone (also migrated from 3.2.9 to 3.3.3) and I also see quite some:

Correctly recompiled closures: 4917. Erred recompiled closures: 30807

And I still see the same scenario.. closures that come from ifEmpty or ifAbsent or things like that that should be gone when method is recompiled... Some of these closures/methods are not even from my app but from tODE. 

With some code to fix the closures I faced while running some parts of the app, I started to make the app to work. I cannot spend much longer testing the general issue but instead try to make the app to work. It's bad because I may lazily find more and more closures needing to recompile as the app receives more testing. 

Anyway, I will do some more tests with findAllReferencePathsToObject: to see if I found the issue...

In case you think there could be something you may want to try / do let me know (compressed backup of this extent is only 80MB).

Thanks for your help!

On Wed, Feb 1, 2017 at 1:20 PM, Dale Henrichs <[hidden email]> wrote:

On 02/01/2017 04:40 AM, Mariano Martinez Peck wrote:

On Wed, Feb 1, 2017 at 2:20 AM, Dale Henrichs <[hidden email]> wrote:

On 1/31/17 7:44 PM, Mariano Martinez Peck wrote:

On Tue, Jan 31, 2017 at 2:25 PM, Dale Henrichs via Glass <[hidden email]> wrote:

Mariano,

1. Find allInstances of ExecBlock

2. Select all instances that #needsRecompile.

3. Send #recompile to the selected instances and collect instances that signal an error during #recompile. The #recompile method attempts to recompile Simple blocks in-place if there is an error during compilation or the block is not simple, an Error is signaled.

4. For the remaining instances of ExecBlock, you will have to track down the reference to the block and reinstall a new, recompiled instance of the ExecBlock.

    1. look at the #blockSource and the #receiver for the block and you might recognize the method needed to recreate the block

    2. look at the #blockSource and pick out a likely chunk of source that you can use to search in method source (`browse source --exact `<source-chunk>`)

    3. Use Repository>>findReferencePathToObject: or Repository>>findAllReferencePathsToObject: to find which object(s) are holding onto the block.

Hi Dale,

OK, it seems I am not having good luck. I tried the following code:

[

| allBlocks allBlocksNeedingRecompile blocksFailedToRecompile correctlyRecompiled | 

blocksFailedToRecompile := OrderedCollection new. 

correctlyRecompiled := OrderedCollection new. 

System commit. 

allBlocks := ExecBlock allSubInstances.

allBlocksNeedingRecompile := allBlocks select: [:each | each needsRecompile ]. 

allBlocksNeedingRecompile do: [:each |

[ 

each recompile.

correctlyRecompiled add: each.

  ] on: Error do: [:ex | 

blocksFailedToRecompile add: each.

]

].

System commit. 

Transcript show:  'Correctly recompiled closures: ',  correctlyRecompiled size asString.

Transcript show: 'Erred recompiled closures: ',  blocksFailedToRecompile size asString.

] on: Warning do: [:ex | ex resume  ]

And it outputs:

Correctly recompiled closures: 0Erred recompiled closures: 797155

Wow. That's a lot to fix. 

But things get strange when I analyze which are the failing closures. For example,  I have an ExecBlock whose source is:

"This is source for a block.  " 

 ^ [ 

     self error:

         'No concrete db roles associated to '''

           , (FaDBAccessor common users row: username column: 'CONCRETE ROLE')

           , ''' of user ''' , username

           ,

             '''. Either the concrete db role id is wrong or the roles must be added to concreteRoles.csv' 

]

That closure, is a closure of an ifEmpty:  of the method #sessionMapFor:prepareAccessLimitingSessionsWith:

See:

sessionMapFor: username prepareAccessLimitingSessionsWith: aDictionaryOfPreparationBlocks

| answer  concreteRole abstractRoleID  |

answer := IdentityDictionary new.

concreteRole := self concreteRoleForUsername: username. 

concreteRole

   ifEmpty: [ 

     self error:

         'No concrete db roles associated to '''

           , (FaDBAccessor common users row: username column: 'CONCRETE ROLE')

           , ''' of user ''' , username

           ,

             '''. Either the concrete db role id is wrong or the roles must be added to concreteRoles.csv' 

].

.....

What is going on? I re-load all my app code after I upgraded stone so I expect that method to have been recompiled. In fact, I just edited the method and saved it again (so that a recompilation was triggered) and the above closure still appears as needing compilation!

Even worst....   the _method  instVar of that example IS NOT IDENTICAL (same OOP) to the one installed in the method dictionary (FaDBSessionsByRolesFactory >> #sessionMapFor:prepareAccessLimitingSessionsWith: ). So it looks like if they were un-installed closures. 

I thought it could be some MC version history around or whatever. So I performed my typical GC and cleanup code which should collect all that garbage. Still same numbers.

mmmm I am running out of ideas. 

Thoughts?

I imagine that the #sessionMapFor:prepareAccessLimitingSessionsWith: method is in an older version of the class, which would explain the differences in oops and the fact that when you recompile the method in the current version of the class, you don't eliminate block that needs to be recompiled ...

`GsDeployer new classesWithClassHistory` should give you the list of classes that have older versions ...

Hi Dale,

I tried with that and it answers an empty array. 

 

Now, I tried in a much smaller stone (also migrated from 3.2.9 to 3.3.3) and I also see quite some:

Correctly recompiled closures: 4917. Erred recompiled closures: 30807

And I still see the same scenario.. closures that come from ifEmpty or ifAbsent or things like that that should be gone when method is recompiled... Some of these closures/methods are not even from my app but from tODE. 

With some code to fix the closures I faced while running some parts of the app, I started to make the app to work. I cannot spend much longer testing the general issue but instead try to make the app to work. It's bad because I may lazily find more and more closures needing to recompile as the app receives more testing. 

Anyway, I will do some more tests with findAllReferencePathsToObject: to see if I found the issue...

In case you think there could be something you may want to try / do let me know (compressed backup of this extent is only 80MB).

Thanks for your help!

I assume that you haven't  done an MFC after the upgrade. At this point it seems likely that you are looking at the old versions of the methods...

This is probably an important step to add to the instructions ... step 0: run an mfc ...

I did that, but I admit it took me some time to realize that could be a problem. In fact, in that very same stone, the 30k decreased to 600 (still quite some to track down!). So yeah, I think such a step 0 MFC would be nice to add to docs. I don't know how to open an issue for that.

Nevertheless, in the other stone (the 50GB one) I keep having a bit less than a million closures even after MFC.  

On Wed, Feb 1, 2017 at 2:24 PM, Dale Henrichs <[hidden email]> wrote:

On 02/01/2017 09:09 AM, Mariano Martinez Peck wrote:

On Wed, Feb 1, 2017 at 1:20 PM, Dale Henrichs <[hidden email]> wrote:

On 02/01/2017 04:40 AM, Mariano Martinez Peck wrote:

On Wed, Feb 1, 2017 at 2:20 AM, Dale Henrichs <[hidden email]> wrote:

On 1/31/17 7:44 PM, Mariano Martinez Peck wrote:

On Tue, Jan 31, 2017 at 2:25 PM, Dale Henrichs via Glass <[hidden email]> wrote:

Mariano,

1. Find allInstances of ExecBlock

2. Select all instances that #needsRecompile.

3. Send #recompile to the selected instances and collect instances that signal an error during #recompile. The #recompile method attempts to recompile Simple blocks in-place if there is an error during compilation or the block is not simple, an Error is signaled.

4. For the remaining instances of ExecBlock, you will have to track down the reference to the block and reinstall a new, recompiled instance of the ExecBlock.

    1. look at the #blockSource and the #receiver for the block and you might recognize the method needed to recreate the block

    2. look at the #blockSource and pick out a likely chunk of source that you can use to search in method source (`browse source --exact `<source-chunk>`)

    3. Use Repository>>findReferencePathToObject: or Repository>>findAllReferencePathsToObject: to find which object(s) are holding onto the block.

Hi Dale,

OK, it seems I am not having good luck. I tried the following code:

[

| allBlocks allBlocksNeedingRecompile blocksFailedToRecompile correctlyRecompiled | 

blocksFailedToRecompile := OrderedCollection new. 

correctlyRecompiled := OrderedCollection new. 

System commit. 

allBlocks := ExecBlock allSubInstances.

allBlocksNeedingRecompile := allBlocks select: [:each | each needsRecompile ]. 

allBlocksNeedingRecompile do: [:each |

[ 

each recompile.

correctlyRecompiled add: each.

  ] on: Error do: [:ex | 

blocksFailedToRecompile add: each.

]

].

System commit. 

Transcript show:  'Correctly recompiled closures: ',  correctlyRecompiled size asString.

Transcript show: 'Erred recompiled closures: ',  blocksFailedToRecompile size asString.

] on: Warning do: [:ex | ex resume  ]

And it outputs:

Correctly recompiled closures: 0Erred recompiled closures: 797155

Wow. That's a lot to fix. 

But things get strange when I analyze which are the failing closures. For example,  I have an ExecBlock whose source is:

"This is source for a block.  " 

 ^ [ 

     self error:

         'No concrete db roles associated to '''

           , (FaDBAccessor common users row: username column: 'CONCRETE ROLE')

           , ''' of user ''' , username

           ,

             '''. Either the concrete db role id is wrong or the roles must be added to concreteRoles.csv' 

]

That closure, is a closure of an ifEmpty:  of the method #sessionMapFor:prepareAccessLimitingSessionsWith:

See:

sessionMapFor: username prepareAccessLimitingSessionsWith: aDictionaryOfPreparationBlocks

| answer  concreteRole abstractRoleID  |

answer := IdentityDictionary new.

concreteRole := self concreteRoleForUsername: username. 

concreteRole

   ifEmpty: [ 

     self error:

         'No concrete db roles associated to '''

           , (FaDBAccessor common users row: username column: 'CONCRETE ROLE')

           , ''' of user ''' , username

           ,

             '''. Either the concrete db role id is wrong or the roles must be added to concreteRoles.csv' 

].

.....

What is going on? I re-load all my app code after I upgraded stone so I expect that method to have been recompiled. In fact, I just edited the method and saved it again (so that a recompilation was triggered) and the above closure still appears as needing compilation!

Even worst....   the _method  instVar of that example IS NOT IDENTICAL (same OOP) to the one installed in the method dictionary (FaDBSessionsByRolesFactory >> #sessionMapFor:prepareAccessLimitingSessionsWith: ). So it looks like if they were un-installed closures. 

I thought it could be some MC version history around or whatever. So I performed my typical GC and cleanup code which should collect all that garbage. Still same numbers.

mmmm I am running out of ideas. 

Thoughts?

I imagine that the #sessionMapFor:prepareAccessLimitingSessionsWith: method is in an older version of the class, which would explain the differences in oops and the fact that when you recompile the method in the current version of the class, you don't eliminate block that needs to be recompiled ...

`GsDeployer new classesWithClassHistory` should give you the list of classes that have older versions ...

Hi Dale,

I tried with that and it answers an empty array. 

 

Now, I tried in a much smaller stone (also migrated from 3.2.9 to 3.3.3) and I also see quite some:

Correctly recompiled closures: 4917. Erred recompiled closures: 30807

And I still see the same scenario.. closures that come from ifEmpty or ifAbsent or things like that that should be gone when method is recompiled... Some of these closures/methods are not even from my app but from tODE. 

With some code to fix the closures I faced while running some parts of the app, I started to make the app to work. I cannot spend much longer testing the general issue but instead try to make the app to work. It's bad because I may lazily find more and more closures needing to recompile as the app receives more testing. 

Anyway, I will do some more tests with findAllReferencePathsToObject: to see if I found the issue...

In case you think there could be something you may want to try / do let me know (compressed backup of this extent is only 80MB).

Thanks for your help!

I assume that you haven't  done an MFC after the upgrade. At this point it seems likely that you are looking at the old versions of the methods...

This is probably an important step to add to the instructions ... step 0: run an mfc ...

I did that, but I admit it took me some time to realize that could be a problem. In fact, in that very same stone, the 30k decreased to 600 (still quite some to track down!). So yeah, I think such a step 0 MFC would be nice to add to docs. I don't know how to open an issue for that.

internal bug #46657 (I already opened one:)

Nevertheless, in the other stone (the 50GB one) I keep having a bit less than a million closures even after MFC.  

I think we need to become a bit more methodical in tracking this down ...  if you've done an all instances run, you should avoid persisting the collection and you should probably logout of the session before doing an MFC.

Yes, I do this. 

 

So I would like to know that there are no classes in class history ...

Well, here i just discovered an issue...

`GsDeployer new classesWithClassHistory`    shows empty (as I told you earlier). 

But... tODE `cls history` shows a bunch!  And in fact, the examples I found in the small stone seem to be from the classes listed in `cls history` ... 

I have to admit that I am not sure I like #classesWithClassHistory to NOT show me certain classes even if they are "core" from GemSTone. I pretty much have them all (like `cls history`) and decide myself which ones are important to me.