Quantcast

Migrating from 3.2.9 to 3.3.3 block compilation issue

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
16 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Migrating from 3.2.9 to 3.3.3 block compilation issue

GLASS mailing list
Hi guys, 

I am migrating some stones (tODE and GsDevKit based) that are on GemStone 3.2.9 to 3.3.3. I am doing this with the `upgradeStone` gsDevKit command. 

I first hit one problem when trying to execute this code via topaz:

MCRepositoryGroup default removeRepository: (MCHttpRepository
                        location: 'xxx'
                        user: ''
                        password: '').



ERROR 2142 , a ImproperOperation occurred (error 2142), Cannot execute method, 'method needs recompile, MCRepositoryGroup class >> default , oop 206736385' (ImproperOperation)

Then, I said...ok, let's connect with tODE and investigate some more (like doing Object _objectForOop: 206736385). But when I try to connect with tODE I get:


Error running testLogin:

'Error: Unable to log into server:a ImproperOperation occurred (error 2142), Cannot execute method, ''method needs recompile, TDTopezServer class >> createFor: , oop 245949953''with: ''TDTopezServer createFor: 1'''

NOTE: netldi log can be found in the file: '$GS_HOME/server/stones/<stone-name>/logs/netldi.log'.



So...looks like I have methods to be recompiled everywhere. I saw this issue [1] but that does not seem to help as these closures are not in sorted collections right ?

Now, manually fixing each, like this:

 | sc sortBlock val cls str val |
  sc := "sortedcollection instance".
  sortBlock := sc sortBlock.
  str := sortBlock _sourceString.
  val := str evaluate.
  sc _sortBlock: val

Is a pain...because I must know how to re-assign the new compiled method. Is there an alternative GENERIC solution to recompile all needed ones? I can think of that but using #become:


System commit. 
  [ 
  MCPlatformSupport installAlmostOutOfMemoryStaticHandler: 75.
  ExecBlock allSubInstances do: [:each | 
each become: each _sourceString evaluate. 
].
]
  ensure: [ MCPlatformSupport uninstallAlmostOutOfMemoryStaticHandler ].
System commit. 

But...comment of #become: says that BlockClosure instances cannot be become...BUUUUUUH. 

So.. any idea?

Thanks in advance,


_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Migrating from 3.2.9 to 3.3.3 block compilation issue

GLASS mailing list
Some more info...

The exact upgareStone I used was:

upgradeStoneName="${sourceStoneName}_333"
$GS_HOME/bin/upgradeStone -f -U "$stoneUsername" -P "$stonePass" "$sourceStoneName" "$upgradeStoneName" 3.3.3

Note that I KNOW I must recompile all methods, that's what I tried to do, but when trying to do so, I get to that error. Maybe I just need to do a few #recompileAllMethods to a concrete number of classes ? (like MCRepositoryGroup)
What I don't understand is why `upgradeStone` did not recompile those automatically. Shouldn't that had happened?

Thanks in advance


On Thu, Jan 26, 2017 at 4:01 PM, Mariano Martinez Peck <[hidden email]> wrote:
Hi guys, 

I am migrating some stones (tODE and GsDevKit based) that are on GemStone 3.2.9 to 3.3.3. I am doing this with the `upgradeStone` gsDevKit command. 

I first hit one problem when trying to execute this code via topaz:

MCRepositoryGroup default removeRepository: (MCHttpRepository
                        location: 'xxx'
                        user: ''
                        password: '').



ERROR 2142 , a ImproperOperation occurred (error 2142), Cannot execute method, 'method needs recompile, MCRepositoryGroup class >> default , oop 206736385' (ImproperOperation)

Then, I said...ok, let's connect with tODE and investigate some more (like doing Object _objectForOop: 206736385). But when I try to connect with tODE I get:


Error running testLogin:

'Error: Unable to log into server:a ImproperOperation occurred (error 2142), Cannot execute method, ''method needs recompile, TDTopezServer class >> createFor: , oop 245949953''with: ''TDTopezServer createFor: 1'''

NOTE: netldi log can be found in the file: '$GS_HOME/server/stones/<stone-name>/logs/netldi.log'.



So...looks like I have methods to be recompiled everywhere. I saw this issue [1] but that does not seem to help as these closures are not in sorted collections right ?

Now, manually fixing each, like this:

 | sc sortBlock val cls str val |
  sc := "sortedcollection instance".
  sortBlock := sc sortBlock.
  str := sortBlock _sourceString.
  val := str evaluate.
  sc _sortBlock: val

Is a pain...because I must know how to re-assign the new compiled method. Is there an alternative GENERIC solution to recompile all needed ones? I can think of that but using #become:


System commit. 
  [ 
  MCPlatformSupport installAlmostOutOfMemoryStaticHandler: 75.
  ExecBlock allSubInstances do: [:each | 
each become: each _sourceString evaluate. 
].
]
  ensure: [ MCPlatformSupport uninstallAlmostOutOfMemoryStaticHandler ].
System commit. 

But...comment of #become: says that BlockClosure instances cannot be become...BUUUUUUH. 

So.. any idea?

Thanks in advance,




--

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Migrating from 3.2.9 to 3.3.3 block compilation issue

GLASS mailing list
In reply to this post by GLASS mailing list

Mariano,

I'll dig into this later this afternoon (I'm out of time right now) ... All of the code is reloaded at each upgrade, so it is likely that there are few blocks that are hanging around (SortedCollections, etc.) that need to be rebuilt ...

Dale

On 01/26/2017 11:01 AM, Mariano Martinez Peck via Glass wrote:
Hi guys, 

I am migrating some stones (tODE and GsDevKit based) that are on GemStone 3.2.9 to 3.3.3. I am doing this with the `upgradeStone` gsDevKit command. 

I first hit one problem when trying to execute this code via topaz:

MCRepositoryGroup default removeRepository: (MCHttpRepository
                        location: 'xxx'
                        user: ''
                        password: '').



ERROR 2142 , a ImproperOperation occurred (error 2142), Cannot execute method, 'method needs recompile, MCRepositoryGroup class >> default , oop 206736385' (ImproperOperation)

Then, I said...ok, let's connect with tODE and investigate some more (like doing Object _objectForOop: 206736385). But when I try to connect with tODE I get:


Error running testLogin:

'Error: Unable to log into server:a ImproperOperation occurred (error 2142), Cannot execute method, ''method needs recompile, TDTopezServer class >> createFor: , oop 245949953''with: ''TDTopezServer createFor: 1'''

NOTE: netldi log can be found in the file: '$GS_HOME/server/stones/<stone-name>/logs/netldi.log'.



So...looks like I have methods to be recompiled everywhere. I saw this issue [1] but that does not seem to help as these closures are not in sorted collections right ?

Now, manually fixing each, like this:

 | sc sortBlock val cls str val |
  sc := "sortedcollection instance".
  sortBlock := sc sortBlock.
  str := sortBlock _sourceString.
  val := str evaluate.
  sc _sortBlock: val

Is a pain...because I must know how to re-assign the new compiled method. Is there an alternative GENERIC solution to recompile all needed ones? I can think of that but using #become:


System commit. 
  [ 
  MCPlatformSupport installAlmostOutOfMemoryStaticHandler: 75.
  ExecBlock allSubInstances do: [:each | 
each become: each _sourceString evaluate. 
].
]
  ensure: [ MCPlatformSupport uninstallAlmostOutOfMemoryStaticHandler ].
System commit. 

But...comment of #become: says that BlockClosure instances cannot be become...BUUUUUUH. 

So.. any idea?

Thanks in advance,



_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass


_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Migrating from 3.2.9 to 3.3.3 block compilation issue

GLASS mailing list
Dale, after checking the logs in upgradeLog directory, I found out that I had errors in file `upgradeTo3x.out ` pasted below.

BTW, I can share with you a zip with all files if you want to. 


Cheers,



topaz 1>
topaz 1> ! clear caches - those that are unneeded or create problem during initial load
topaz 1> run
"MCDefinition clearInstances"
MCDefinition _classVars at: #Instances put: nil.
"MCMethodDefinition shutdown"
MCMethodDefinition _classVars at: #Definitions put: nil.
"MethodVersionHistory reset"
UserGlobals removeKey: #MethodVersionHistory_UniqueInstance ifAbsent: [].
"clear out PackageOrganizer default class instance variable"
PackageOrganizer atClassInstVar: #default put: nil.
"clear out McCacheRepository default class instance variable"
MCCacheRepository atClassInstVar: #default put: nil.
"wipe out the working copy registry ... will be redefined later"
MCPackageManager atClassInstVar: #registry put: nil.
MCWorkingCopy atClassInstVar: #registry put: nil.
"wipe out the default MCRepositoryGroup"
MCRepositoryGroup atClassInstVar: #default put: nil.
"Wipe out the Metacello registry ... if it exists"
(GsCurrentSession currentSession symbolList objectNamed: #MetacelloProjectRegistration)
  ifNotNil: [:cl | cl _classVars at: #Registry put: nil ].
true
%
GemStone Smalltalk Compiler Errors:
   "MCDefinition clearInstances"
   MCDefinition _classVars at: #Instances put: nil.
 * ^1                                                                 *******
   "MCMethodDefinition shutdown"
   MCMethodDefinition _classVars at: #Definitions put: nil.
 * ^2                                                                 *******
   "MethodVersionHistory reset"
   UserGlobals removeKey: #MethodVersionHistory_UniqueInstance ifAbsent: [].
   "clear out PackageOrganizer default class instance variable"
   PackageOrganizer atClassInstVar: #default put: nil.
 * ^3                                                                 *******
   "clear out McCacheRepository default class instance variable"
   MCCacheRepository atClassInstVar: #default put: nil.
 * ^4                                                                 *******
   "wipe out the working copy registry ... will be redefined later"
   MCPackageManager atClassInstVar: #registry put: nil.
 * ^5                                                                 *******
   MCWorkingCopy atClassInstVar: #registry put: nil.
 * ^6                                                                 *******
   "wipe out the default MCRepositoryGroup"
   MCRepositoryGroup atClassInstVar: #default put: nil.
 * ^7                                                                 *******
   "Wipe out the Metacello registry ... if it exists"
   (GsCurrentSession currentSession symbolList objectNamed: #MetacelloProjectRegistration)
     ifNotNil: [:cl | cl _classVars at: #Registry put: nil ].
   true

1: [1031] undefined symbol
2: [1031] undefined symbol
3: [1031] undefined symbol
4: [1031] undefined symbol
5: [1031] undefined symbol
6: [1031] undefined symbol
7: [1031] undefined symbol
ERROR: UNEXPECTED ERROR
topaz> time
 01/26/2017 11:50:27.993 EST
topaz > exec iferr 1 : stk
Stack is not active
topaz > exec iferr 2 : stack
Stack is not active
topaz > exec iferr 3 : input pop
topaz > exec iferr 4 : exit
Logging out session 1.



On Thu, Jan 26, 2017 at 6:11 PM, Dale Henrichs via Glass <[hidden email]> wrote:

Mariano,

I'll dig into this later this afternoon (I'm out of time right now) ... All of the code is reloaded at each upgrade, so it is likely that there are few blocks that are hanging around (SortedCollections, etc.) that need to be rebuilt ...

Dale


On 01/26/2017 11:01 AM, Mariano Martinez Peck via Glass wrote:
Hi guys, 

I am migrating some stones (tODE and GsDevKit based) that are on GemStone 3.2.9 to 3.3.3. I am doing this with the `upgradeStone` gsDevKit command. 

I first hit one problem when trying to execute this code via topaz:

MCRepositoryGroup default removeRepository: (MCHttpRepository
                        location: 'xxx'
                        user: ''
                        password: '').



ERROR 2142 , a ImproperOperation occurred (error 2142), Cannot execute method, 'method needs recompile, MCRepositoryGroup class >> default , oop 206736385' (ImproperOperation)

Then, I said...ok, let's connect with tODE and investigate some more (like doing Object _objectForOop: 206736385). But when I try to connect with tODE I get:


Error running testLogin:

'Error: Unable to log into server:a ImproperOperation occurred (error 2142), Cannot execute method, ''method needs recompile, TDTopezServer class >> createFor: , oop 245949953''with: ''TDTopezServer createFor: 1'''

NOTE: netldi log can be found in the file: '$GS_HOME/server/stones/<stone-name>/logs/netldi.log'.



So...looks like I have methods to be recompiled everywhere. I saw this issue [1] but that does not seem to help as these closures are not in sorted collections right ?

Now, manually fixing each, like this:

 | sc sortBlock val cls str val |
  sc := "sortedcollection instance".
  sortBlock := sc sortBlock.
  str := sortBlock _sourceString.
  val := str evaluate.
  sc _sortBlock: val

Is a pain...because I must know how to re-assign the new compiled method. Is there an alternative GENERIC solution to recompile all needed ones? I can think of that but using #become:


System commit. 
  [ 
  MCPlatformSupport installAlmostOutOfMemoryStaticHandler: 75.
  ExecBlock allSubInstances do: [:each | 
each become: each _sourceString evaluate. 
].
]
  ensure: [ MCPlatformSupport uninstallAlmostOutOfMemoryStaticHandler ].
System commit. 

But...comment of #become: says that BlockClosure instances cannot be become...BUUUUUUH. 

So.. any idea?

Thanks in advance,



_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass


_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass




--

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Migrating from 3.2.9 to 3.3.3 block compilation issue

GLASS mailing list

Mariano,

After looking at your log files, it looks like this is a bug in upgradeStone script that is specific to the gs_port branch --- related to using a non-DataCurator user for GsDevKit --- I've updated the gs_port branch with my proposed fix ...

Dale


On 1/26/17 1:20 PM, Mariano Martinez Peck wrote:
Dale, after checking the logs in upgradeLog directory, I found out that I had errors in file `upgradeTo3x.out ` pasted below.

BTW, I can share with you a zip with all files if you want to. 


Cheers,



topaz 1>
topaz 1> ! clear caches - those that are unneeded or create problem during initial load
topaz 1> run
"MCDefinition clearInstances"
MCDefinition _classVars at: #Instances put: nil.
"MCMethodDefinition shutdown"
MCMethodDefinition _classVars at: #Definitions put: nil.
"MethodVersionHistory reset"
UserGlobals removeKey: #MethodVersionHistory_UniqueInstance ifAbsent: [].
"clear out PackageOrganizer default class instance variable"
PackageOrganizer atClassInstVar: #default put: nil.
"clear out McCacheRepository default class instance variable"
MCCacheRepository atClassInstVar: #default put: nil.
"wipe out the working copy registry ... will be redefined later"
MCPackageManager atClassInstVar: #registry put: nil.
MCWorkingCopy atClassInstVar: #registry put: nil.
"wipe out the default MCRepositoryGroup"
MCRepositoryGroup atClassInstVar: #default put: nil.
"Wipe out the Metacello registry ... if it exists"
(GsCurrentSession currentSession symbolList objectNamed: #MetacelloProjectRegistration)
  ifNotNil: [:cl | cl _classVars at: #Registry put: nil ].
true
%
GemStone Smalltalk Compiler Errors:
   "MCDefinition clearInstances"
   MCDefinition _classVars at: #Instances put: nil.
 * ^1                                                                 *******
   "MCMethodDefinition shutdown"
   MCMethodDefinition _classVars at: #Definitions put: nil.
 * ^2                                                                 *******
   "MethodVersionHistory reset"
   UserGlobals removeKey: #MethodVersionHistory_UniqueInstance ifAbsent: [].
   "clear out PackageOrganizer default class instance variable"
   PackageOrganizer atClassInstVar: #default put: nil.
 * ^3                                                                 *******
   "clear out McCacheRepository default class instance variable"
   MCCacheRepository atClassInstVar: #default put: nil.
 * ^4                                                                 *******
   "wipe out the working copy registry ... will be redefined later"
   MCPackageManager atClassInstVar: #registry put: nil.
 * ^5                                                                 *******
   MCWorkingCopy atClassInstVar: #registry put: nil.
 * ^6                                                                 *******
   "wipe out the default MCRepositoryGroup"
   MCRepositoryGroup atClassInstVar: #default put: nil.
 * ^7                                                                 *******
   "Wipe out the Metacello registry ... if it exists"
   (GsCurrentSession currentSession symbolList objectNamed: #MetacelloProjectRegistration)
     ifNotNil: [:cl | cl _classVars at: #Registry put: nil ].
   true

1: [1031] undefined symbol
2: [1031] undefined symbol
3: [1031] undefined symbol
4: [1031] undefined symbol
5: [1031] undefined symbol
6: [1031] undefined symbol
7: [1031] undefined symbol
ERROR: UNEXPECTED ERROR
topaz> time
 01/26/2017 11:50:27.993 EST
topaz > exec iferr 1 : stk
Stack is not active
topaz > exec iferr 2 : stack
Stack is not active
topaz > exec iferr 3 : input pop
topaz > exec iferr 4 : exit
Logging out session 1.



On Thu, Jan 26, 2017 at 6:11 PM, Dale Henrichs via Glass <[hidden email]> wrote:

Mariano,

I'll dig into this later this afternoon (I'm out of time right now) ... All of the code is reloaded at each upgrade, so it is likely that there are few blocks that are hanging around (SortedCollections, etc.) that need to be rebuilt ...

Dale


On 01/26/2017 11:01 AM, Mariano Martinez Peck via Glass wrote:
Hi guys, 

I am migrating some stones (tODE and GsDevKit based) that are on GemStone 3.2.9 to 3.3.3. I am doing this with the `upgradeStone` gsDevKit command. 

I first hit one problem when trying to execute this code via topaz:

MCRepositoryGroup default removeRepository: (MCHttpRepository
                        location: 'xxx'
                        user: ''
                        password: '').



ERROR 2142 , a ImproperOperation occurred (error 2142), Cannot execute method, 'method needs recompile, MCRepositoryGroup class >> default , oop 206736385' (ImproperOperation)

Then, I said...ok, let's connect with tODE and investigate some more (like doing Object _objectForOop: 206736385). But when I try to connect with tODE I get:


Error running testLogin:

'Error: Unable to log into server:a ImproperOperation occurred (error 2142), Cannot execute method, ''method needs recompile, TDTopezServer class >> createFor: , oop 245949953''with: ''TDTopezServer createFor: 1'''

NOTE: netldi log can be found in the file: '$GS_HOME/server/stones/<stone-name>/logs/netldi.log'.



So...looks like I have methods to be recompiled everywhere. I saw this issue [1] but that does not seem to help as these closures are not in sorted collections right ?

Now, manually fixing each, like this:

 | sc sortBlock val cls str val |
  sc := "sortedcollection instance".
  sortBlock := sc sortBlock.
  str := sortBlock _sourceString.
  val := str evaluate.
  sc _sortBlock: val

Is a pain...because I must know how to re-assign the new compiled method. Is there an alternative GENERIC solution to recompile all needed ones? I can think of that but using #become:


System commit. 
  [ 
  MCPlatformSupport installAlmostOutOfMemoryStaticHandler: 75.
  ExecBlock allSubInstances do: [:each | 
each become: each _sourceString evaluate. 
].
]
  ensure: [ MCPlatformSupport uninstallAlmostOutOfMemoryStaticHandler ].
System commit. 

But...comment of #become: says that BlockClosure instances cannot be become...BUUUUUUH. 

So.. any idea?

Thanks in advance,



_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
_______________________________________________ Glass mailing list [hidden email] http://lists.gemtalksystems.com/mailman/listinfo/glass
--

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Migrating from 3.2.9 to 3.3.3 block compilation issue

GLASS mailing list
In reply to this post by GLASS mailing list
Hi guys,

Ok, as Dale found out, some automatic recompilation from `upgradeStone` was not being done because of a GsDevKit_issue.
Now, problem is that even with that fixed, I am still finding here and there more and more closures that need recompilation. These are NOT sort blocks of sorted collections nor method. They are closures (from my domain objects) stored somewhere. 

Now, manually fixing each, like this:

anObject setter: anObject getter _sourceString evaluate.

Is a pain and not accurate (the closure may be still referenced from another place)...because I must know how to re-assign the new closure. Is there an alternative GENERIC solution to recompile all needed ones? I can think of that but using #become:


System commit. 
  [ 
  MCPlatformSupport installAlmostOutOfMemoryStaticHandler: 75.
    ExecBlock allSubInstances do: [:each | 
each become: each _sourceString evaluate. 
].
]
  ensure: [ MCPlatformSupport uninstallAlmostOutOfMemoryStaticHandler ].
System commit. 

But...comment of #become: says that BlockClosure instances cannot be become...BUUUUUUH. 

So.. any idea?

Thanks in advance,





On Thu, Jan 26, 2017 at 4:01 PM, Mariano Martinez Peck <[hidden email]> wrote:
Hi guys, 

I am migrating some stones (tODE and GsDevKit based) that are on GemStone 3.2.9 to 3.3.3. I am doing this with the `upgradeStone` gsDevKit command. 

I first hit one problem when trying to execute this code via topaz:

MCRepositoryGroup default removeRepository: (MCHttpRepository
                        location: 'xxx'
                        user: ''
                        password: '').



ERROR 2142 , a ImproperOperation occurred (error 2142), Cannot execute method, 'method needs recompile, MCRepositoryGroup class >> default , oop 206736385' (ImproperOperation)

Then, I said...ok, let's connect with tODE and investigate some more (like doing Object _objectForOop: 206736385). But when I try to connect with tODE I get:


Error running testLogin:

'Error: Unable to log into server:a ImproperOperation occurred (error 2142), Cannot execute method, ''method needs recompile, TDTopezServer class >> createFor: , oop 245949953''with: ''TDTopezServer createFor: 1'''

NOTE: netldi log can be found in the file: '$GS_HOME/server/stones/<stone-name>/logs/netldi.log'.



So...looks like I have methods to be recompiled everywhere. I saw this issue [1] but that does not seem to help as these closures are not in sorted collections right ?

Now, manually fixing each, like this:

 | sc sortBlock val cls str val |
  sc := "sortedcollection instance".
  sortBlock := sc sortBlock.
  str := sortBlock _sourceString.
  val := str evaluate.
  sc _sortBlock: val

Is a pain...because I must know how to re-assign the new compiled method. Is there an alternative GENERIC solution to recompile all needed ones? I can think of that but using #become:


System commit. 
  [ 
  MCPlatformSupport installAlmostOutOfMemoryStaticHandler: 75.
  ExecBlock allSubInstances do: [:each | 
each become: each _sourceString evaluate. 
].
]
  ensure: [ MCPlatformSupport uninstallAlmostOutOfMemoryStaticHandler ].
System commit. 

But...comment of #become: says that BlockClosure instances cannot be become...BUUUUUUH. 

So.. any idea?

Thanks in advance,




--

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Migrating from 3.2.9 to 3.3.3 block compilation issue

GLASS mailing list

Mariano,

1. Find allInstances of ExecBlock

2. Select all instances that #needsRecompile.

3. Send #recompile to the selected instances and collect instances that signal an error during #recompile. The #recompile method attempts to recompile Simple blocks in-place if there is an error during compilation or the block is not simple, an Error is signaled.

4. For the remaining instances of ExecBlock, you will have to track down the reference to the block and reinstall a new, recompiled instance of the ExecBlock.

    1. look at the #blockSource and the #receiver for the block and you might recognize the method needed to recreate the block

    2. look at the #blockSource and pick out a likely chunk of source that you can use to search in method source (`browse source --exact `<source-chunk>`)

    3. Use Repository>>findReferencePathToObject: or Repository>>findAllReferencePathsToObject: to find which object(s) are holding onto the block.

Dale

On 01/30/2017 07:15 PM, Mariano Martinez Peck via Glass wrote:
Hi guys,

Ok, as Dale found out, some automatic recompilation from `upgradeStone` was not being done because of a GsDevKit_issue.
Now, problem is that even with that fixed, I am still finding here and there more and more closures that need recompilation. These are NOT sort blocks of sorted collections nor method. They are closures (from my domain objects) stored somewhere. 

Now, manually fixing each, like this:

anObject setter: anObject getter _sourceString evaluate.

Is a pain and not accurate (the closure may be still referenced from another place)...because I must know how to re-assign the new closure. Is there an alternative GENERIC solution to recompile all needed ones? I can think of that but using #become:


System commit. 
  [ 
  MCPlatformSupport installAlmostOutOfMemoryStaticHandler: 75.
    ExecBlock allSubInstances do: [:each | 
each become: each _sourceString evaluate. 
].
]
  ensure: [ MCPlatformSupport uninstallAlmostOutOfMemoryStaticHandler ].
System commit. 

But...comment of #become: says that BlockClosure instances cannot be become...BUUUUUUH. 

So.. any idea?

Thanks in advance,





On Thu, Jan 26, 2017 at 4:01 PM, Mariano Martinez Peck <[hidden email]> wrote:
Hi guys, 

I am migrating some stones (tODE and GsDevKit based) that are on GemStone 3.2.9 to 3.3.3. I am doing this with the `upgradeStone` gsDevKit command. 

I first hit one problem when trying to execute this code via topaz:

MCRepositoryGroup default removeRepository: (MCHttpRepository
                        location: 'xxx'
                        user: ''
                        password: '').



ERROR 2142 , a ImproperOperation occurred (error 2142), Cannot execute method, 'method needs recompile, MCRepositoryGroup class >> default , oop 206736385' (ImproperOperation)

Then, I said...ok, let's connect with tODE and investigate some more (like doing Object _objectForOop: 206736385). But when I try to connect with tODE I get:


Error running testLogin:

'Error: Unable to log into server:a ImproperOperation occurred (error 2142), Cannot execute method, ''method needs recompile, TDTopezServer class >> createFor: , oop 245949953''with: ''TDTopezServer createFor: 1'''

NOTE: netldi log can be found in the file: '$GS_HOME/server/stones/<stone-name>/logs/netldi.log'.



So...looks like I have methods to be recompiled everywhere. I saw this issue [1] but that does not seem to help as these closures are not in sorted collections right ?

Now, manually fixing each, like this:

 | sc sortBlock val cls str val |
  sc := "sortedcollection instance".
  sortBlock := sc sortBlock.
  str := sortBlock _sourceString.
  val := str evaluate.
  sc _sortBlock: val

Is a pain...because I must know how to re-assign the new compiled method. Is there an alternative GENERIC solution to recompile all needed ones? I can think of that but using #become:


System commit. 
  [ 
  MCPlatformSupport installAlmostOutOfMemoryStaticHandler: 75.
  ExecBlock allSubInstances do: [:each | 
each become: each _sourceString evaluate. 
].
]
  ensure: [ MCPlatformSupport uninstallAlmostOutOfMemoryStaticHandler ].
System commit. 

But...comment of #become: says that BlockClosure instances cannot be become...BUUUUUUH. 

So.. any idea?

Thanks in advance,




--


_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass


_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Migrating from 3.2.9 to 3.3.3 block compilation issue

GLASS mailing list


On Tue, Jan 31, 2017 at 2:25 PM, Dale Henrichs via Glass <[hidden email]> wrote:

Mariano,

1. Find allInstances of ExecBlock

2. Select all instances that #needsRecompile.

3. Send #recompile to the selected instances and collect instances that signal an error during #recompile. The #recompile method attempts to recompile Simple blocks in-place if there is an error during compilation or the block is not simple, an Error is signaled.

4. For the remaining instances of ExecBlock, you will have to track down the reference to the block and reinstall a new, recompiled instance of the ExecBlock.

    1. look at the #blockSource and the #receiver for the block and you might recognize the method needed to recreate the block

    2. look at the #blockSource and pick out a likely chunk of source that you can use to search in method source (`browse source --exact `<source-chunk>`)

    3. Use Repository>>findReferencePathToObject: or Repository>>findAllReferencePathsToObject: to find which object(s) are holding onto the block.


Hi Dale,

OK, it seems I am not having good luck. I tried the following code:

[
| allBlocks allBlocksNeedingRecompile blocksFailedToRecompile correctlyRecompiled | 
blocksFailedToRecompile := OrderedCollection new. 
correctlyRecompiled := OrderedCollection new. 
System commit. 
allBlocks := ExecBlock allSubInstances.
allBlocksNeedingRecompile := allBlocks select: [:each | each needsRecompile ]. 
allBlocksNeedingRecompile do: [:each |
each recompile.
correctlyRecompiled add: each.
  ] on: Error do: [:ex | 
blocksFailedToRecompile add: each.
]
].
System commit. 
Transcript show:  'Correctly recompiled closures: ',  correctlyRecompiled size asString.
Transcript show: 'Erred recompiled closures: ',  blocksFailedToRecompile size asString.
] on: Warning do: [:ex | ex resume  ]


And it outputs:

Correctly recompiled closures: 0Erred recompiled closures: 797155

Wow. That's a lot to fix. 

But things get strange when I analyze which are the failing closures. For example,  I have an ExecBlock whose source is:

"This is source for a block.  " 
 ^ [ 
     self error:
         'No concrete db roles associated to '''
           , (FaDBAccessor common users row: username column: 'CONCRETE ROLE')
           , ''' of user ''' , username
           ,
             '''. Either the concrete db role id is wrong or the roles must be added to concreteRoles.csv' 
]



That closure, is a closure of an ifEmpty:  of the method #sessionMapFor:prepareAccessLimitingSessionsWith:
See:

sessionMapFor: username prepareAccessLimitingSessionsWith: aDictionaryOfPreparationBlocks
| answer  concreteRole abstractRoleID  |
answer := IdentityDictionary new.
concreteRole := self concreteRoleForUsername: username. 
concreteRole
   ifEmpty: [ 
     self error:
         'No concrete db roles associated to '''
           , (FaDBAccessor common users row: username column: 'CONCRETE ROLE')
           , ''' of user ''' , username
           ,
             '''. Either the concrete db role id is wrong or the roles must be added to concreteRoles.csv' 
].
.....


What is going on? I re-load all my app code after I upgraded stone so I expect that method to have been recompiled. In fact, I just edited the method and saved it again (so that a recompilation was triggered) and the above closure still appears as needing compilation!

Even worst....   the _method  instVar of that example IS NOT IDENTICAL (same OOP) to the one installed in the method dictionary (FaDBSessionsByRolesFactory >> #sessionMapFor:prepareAccessLimitingSessionsWith: ). So it looks like if they were un-installed closures. 

I thought it could be some MC version history around or whatever. So I performed my typical GC and cleanup code which should collect all that garbage. Still same numbers.

mmmm I am running out of ideas. 

Thoughts?

Thanks in advance



 

Dale

On 01/30/2017 07:15 PM, Mariano Martinez Peck via Glass wrote:
Hi guys,

Ok, as Dale found out, some automatic recompilation from `upgradeStone` was not being done because of a GsDevKit_issue.
Now, problem is that even with that fixed, I am still finding here and there more and more closures that need recompilation. These are NOT sort blocks of sorted collections nor method. They are closures (from my domain objects) stored somewhere. 

Now, manually fixing each, like this:

anObject setter: anObject getter _sourceString evaluate.

Is a pain and not accurate (the closure may be still referenced from another place)...because I must know how to re-assign the new closure. Is there an alternative GENERIC solution to recompile all needed ones? I can think of that but using #become:


System commit. 
  [ 
  MCPlatformSupport installAlmostOutOfMemoryStaticHandler: 75.
    ExecBlock allSubInstances do: [:each | 
each become: each _sourceString evaluate. 
].
]
  ensure: [ MCPlatformSupport uninstallAlmostOutOfMemoryStaticHandler ].
System commit. 

But...comment of #become: says that BlockClosure instances cannot be become...BUUUUUUH. 

So.. any idea?

Thanks in advance,





On Thu, Jan 26, 2017 at 4:01 PM, Mariano Martinez Peck <[hidden email]> wrote:
Hi guys, 

I am migrating some stones (tODE and GsDevKit based) that are on GemStone 3.2.9 to 3.3.3. I am doing this with the `upgradeStone` gsDevKit command. 

I first hit one problem when trying to execute this code via topaz:

MCRepositoryGroup default removeRepository: (MCHttpRepository
                        location: 'xxx'
                        user: ''
                        password: '').



ERROR 2142 , a ImproperOperation occurred (error 2142), Cannot execute method, 'method needs recompile, MCRepositoryGroup class >> default , oop 206736385' (ImproperOperation)

Then, I said...ok, let's connect with tODE and investigate some more (like doing Object _objectForOop: 206736385). But when I try to connect with tODE I get:


Error running testLogin:

'Error: Unable to log into server:a ImproperOperation occurred (error 2142), Cannot execute method, ''method needs recompile, TDTopezServer class >> createFor: , oop 245949953''with: ''TDTopezServer createFor: 1'''

NOTE: netldi log can be found in the file: '$GS_HOME/server/stones/<stone-name>/logs/netldi.log'.



So...looks like I have methods to be recompiled everywhere. I saw this issue [1] but that does not seem to help as these closures are not in sorted collections right ?

Now, manually fixing each, like this:

 | sc sortBlock val cls str val |
  sc := "sortedcollection instance".
  sortBlock := sc sortBlock.
  str := sortBlock _sourceString.
  val := str evaluate.
  sc _sortBlock: val

Is a pain...because I must know how to re-assign the new compiled method. Is there an alternative GENERIC solution to recompile all needed ones? I can think of that but using #become:


System commit. 
  [ 
  MCPlatformSupport installAlmostOutOfMemoryStaticHandler: 75.
  ExecBlock allSubInstances do: [:each | 
each become: each _sourceString evaluate. 
].
]
  ensure: [ MCPlatformSupport uninstallAlmostOutOfMemoryStaticHandler ].
System commit. 

But...comment of #become: says that BlockClosure instances cannot be become...BUUUUUUH. 

So.. any idea?

Thanks in advance,




--


_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass


_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass




--

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Migrating from 3.2.9 to 3.3.3 block compilation issue

GLASS mailing list
BTW... i forgot to say... the on: Warning do:   is because else i get:

Warning: cannot find key for optimized literal value oop 655362 (a SmallInteger) 

For many many many closures (I think I get this when I ask #needsRecompile or #recompile (I don't remember).

Just in case this is useful...

On Wed, Feb 1, 2017 at 12:44 AM, Mariano Martinez Peck <[hidden email]> wrote:


On Tue, Jan 31, 2017 at 2:25 PM, Dale Henrichs via Glass <[hidden email]> wrote:

Mariano,

1. Find allInstances of ExecBlock

2. Select all instances that #needsRecompile.

3. Send #recompile to the selected instances and collect instances that signal an error during #recompile. The #recompile method attempts to recompile Simple blocks in-place if there is an error during compilation or the block is not simple, an Error is signaled.

4. For the remaining instances of ExecBlock, you will have to track down the reference to the block and reinstall a new, recompiled instance of the ExecBlock.

    1. look at the #blockSource and the #receiver for the block and you might recognize the method needed to recreate the block

    2. look at the #blockSource and pick out a likely chunk of source that you can use to search in method source (`browse source --exact `<source-chunk>`)

    3. Use Repository>>findReferencePathToObject: or Repository>>findAllReferencePathsToObject: to find which object(s) are holding onto the block.


Hi Dale,

OK, it seems I am not having good luck. I tried the following code:

[
| allBlocks allBlocksNeedingRecompile blocksFailedToRecompile correctlyRecompiled | 
blocksFailedToRecompile := OrderedCollection new. 
correctlyRecompiled := OrderedCollection new. 
System commit. 
allBlocks := ExecBlock allSubInstances.
allBlocksNeedingRecompile := allBlocks select: [:each | each needsRecompile ]. 
allBlocksNeedingRecompile do: [:each |
each recompile.
correctlyRecompiled add: each.
  ] on: Error do: [:ex | 
blocksFailedToRecompile add: each.
]
].
System commit. 
Transcript show:  'Correctly recompiled closures: ',  correctlyRecompiled size asString.
Transcript show: 'Erred recompiled closures: ',  blocksFailedToRecompile size asString.
] on: Warning do: [:ex | ex resume  ]


And it outputs:

Correctly recompiled closures: 0Erred recompiled closures: 797155

Wow. That's a lot to fix. 

But things get strange when I analyze which are the failing closures. For example,  I have an ExecBlock whose source is:

"This is source for a block.  " 
 ^ [ 
     self error:
         'No concrete db roles associated to '''
           , (FaDBAccessor common users row: username column: 'CONCRETE ROLE')
           , ''' of user ''' , username
           ,
             '''. Either the concrete db role id is wrong or the roles must be added to concreteRoles.csv' 
]



That closure, is a closure of an ifEmpty:  of the method #sessionMapFor:prepareAccessLimitingSessionsWith:
See:

sessionMapFor: username prepareAccessLimitingSessionsWith: aDictionaryOfPreparationBlocks
| answer  concreteRole abstractRoleID  |
answer := IdentityDictionary new.
concreteRole := self concreteRoleForUsername: username. 
concreteRole
   ifEmpty: [ 
     self error:
         'No concrete db roles associated to '''
           , (FaDBAccessor common users row: username column: 'CONCRETE ROLE')
           , ''' of user ''' , username
           ,
             '''. Either the concrete db role id is wrong or the roles must be added to concreteRoles.csv' 
].
.....


What is going on? I re-load all my app code after I upgraded stone so I expect that method to have been recompiled. In fact, I just edited the method and saved it again (so that a recompilation was triggered) and the above closure still appears as needing compilation!

Even worst....   the _method  instVar of that example IS NOT IDENTICAL (same OOP) to the one installed in the method dictionary (FaDBSessionsByRolesFactory >> #sessionMapFor:prepareAccessLimitingSessionsWith: ). So it looks like if they were un-installed closures. 

I thought it could be some MC version history around or whatever. So I performed my typical GC and cleanup code which should collect all that garbage. Still same numbers.

mmmm I am running out of ideas. 

Thoughts?

Thanks in advance



 

Dale

On 01/30/2017 07:15 PM, Mariano Martinez Peck via Glass wrote:
Hi guys,

Ok, as Dale found out, some automatic recompilation from `upgradeStone` was not being done because of a GsDevKit_issue.
Now, problem is that even with that fixed, I am still finding here and there more and more closures that need recompilation. These are NOT sort blocks of sorted collections nor method. They are closures (from my domain objects) stored somewhere. 

Now, manually fixing each, like this:

anObject setter: anObject getter _sourceString evaluate.

Is a pain and not accurate (the closure may be still referenced from another place)...because I must know how to re-assign the new closure. Is there an alternative GENERIC solution to recompile all needed ones? I can think of that but using #become:


System commit. 
  [ 
  MCPlatformSupport installAlmostOutOfMemoryStaticHandler: 75.
    ExecBlock allSubInstances do: [:each | 
each become: each _sourceString evaluate. 
].
]
  ensure: [ MCPlatformSupport uninstallAlmostOutOfMemoryStaticHandler ].
System commit. 

But...comment of #become: says that BlockClosure instances cannot be become...BUUUUUUH. 

So.. any idea?

Thanks in advance,





On Thu, Jan 26, 2017 at 4:01 PM, Mariano Martinez Peck <[hidden email]> wrote:
Hi guys, 

I am migrating some stones (tODE and GsDevKit based) that are on GemStone 3.2.9 to 3.3.3. I am doing this with the `upgradeStone` gsDevKit command. 

I first hit one problem when trying to execute this code via topaz:

MCRepositoryGroup default removeRepository: (MCHttpRepository
                        location: 'xxx'
                        user: ''
                        password: '').



ERROR 2142 , a ImproperOperation occurred (error 2142), Cannot execute method, 'method needs recompile, MCRepositoryGroup class >> default , oop 206736385' (ImproperOperation)

Then, I said...ok, let's connect with tODE and investigate some more (like doing Object _objectForOop: 206736385). But when I try to connect with tODE I get:


Error running testLogin:

'Error: Unable to log into server:a ImproperOperation occurred (error 2142), Cannot execute method, ''method needs recompile, TDTopezServer class >> createFor: , oop 245949953''with: ''TDTopezServer createFor: 1'''

NOTE: netldi log can be found in the file: '$GS_HOME/server/stones/<stone-name>/logs/netldi.log'.



So...looks like I have methods to be recompiled everywhere. I saw this issue [1] but that does not seem to help as these closures are not in sorted collections right ?

Now, manually fixing each, like this:

 | sc sortBlock val cls str val |
  sc := "sortedcollection instance".
  sortBlock := sc sortBlock.
  str := sortBlock _sourceString.
  val := str evaluate.
  sc _sortBlock: val

Is a pain...because I must know how to re-assign the new compiled method. Is there an alternative GENERIC solution to recompile all needed ones? I can think of that but using #become:


System commit. 
  [ 
  MCPlatformSupport installAlmostOutOfMemoryStaticHandler: 75.
  ExecBlock allSubInstances do: [:each | 
each become: each _sourceString evaluate. 
].
]
  ensure: [ MCPlatformSupport uninstallAlmostOutOfMemoryStaticHandler ].
System commit. 

But...comment of #become: says that BlockClosure instances cannot be become...BUUUUUUH. 

So.. any idea?

Thanks in advance,




--


_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass


_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass




--



--

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Migrating from 3.2.9 to 3.3.3 block compilation issue

GLASS mailing list
In reply to this post by GLASS mailing list
On 1/31/17 7:44 PM, Mariano Martinez Peck wrote:


On Tue, Jan 31, 2017 at 2:25 PM, Dale Henrichs via Glass <[hidden email]> wrote:

Mariano,

1. Find allInstances of ExecBlock

2. Select all instances that #needsRecompile.

3. Send #recompile to the selected instances and collect instances that signal an error during #recompile. The #recompile method attempts to recompile Simple blocks in-place if there is an error during compilation or the block is not simple, an Error is signaled.

4. For the remaining instances of ExecBlock, you will have to track down the reference to the block and reinstall a new, recompiled instance of the ExecBlock.

    1. look at the #blockSource and the #receiver for the block and you might recognize the method needed to recreate the block

    2. look at the #blockSource and pick out a likely chunk of source that you can use to search in method source (`browse source --exact `<source-chunk>`)

    3. Use Repository>>findReferencePathToObject: or Repository>>findAllReferencePathsToObject: to find which object(s) are holding onto the block.


Hi Dale,

OK, it seems I am not having good luck. I tried the following code:

[
| allBlocks allBlocksNeedingRecompile blocksFailedToRecompile correctlyRecompiled | 
blocksFailedToRecompile := OrderedCollection new. 
correctlyRecompiled := OrderedCollection new. 
System commit. 
allBlocks := ExecBlock allSubInstances.
allBlocksNeedingRecompile := allBlocks select: [:each | each needsRecompile ]. 
allBlocksNeedingRecompile do: [:each |
each recompile.
correctlyRecompiled add: each.
  ] on: Error do: [:ex | 
blocksFailedToRecompile add: each.
]
].
System commit. 
Transcript show:  'Correctly recompiled closures: ',  correctlyRecompiled size asString.
Transcript show: 'Erred recompiled closures: ',  blocksFailedToRecompile size asString.
] on: Warning do: [:ex | ex resume  ]


And it outputs:

Correctly recompiled closures: 0Erred recompiled closures: 797155

Wow. That's a lot to fix. 

But things get strange when I analyze which are the failing closures. For example,  I have an ExecBlock whose source is:

"This is source for a block.  " 
 ^ [ 
     self error:
         'No concrete db roles associated to '''
           , (FaDBAccessor common users row: username column: 'CONCRETE ROLE')
           , ''' of user ''' , username
           ,
             '''. Either the concrete db role id is wrong or the roles must be added to concreteRoles.csv' 
]



That closure, is a closure of an ifEmpty:  of the method #sessionMapFor:prepareAccessLimitingSessionsWith:
See:

sessionMapFor: username prepareAccessLimitingSessionsWith: aDictionaryOfPreparationBlocks
| answer  concreteRole abstractRoleID  |
answer := IdentityDictionary new.
concreteRole := self concreteRoleForUsername: username. 
concreteRole
   ifEmpty: [ 
     self error:
         'No concrete db roles associated to '''
           , (FaDBAccessor common users row: username column: 'CONCRETE ROLE')
           , ''' of user ''' , username
           ,
             '''. Either the concrete db role id is wrong or the roles must be added to concreteRoles.csv' 
].
.....


What is going on? I re-load all my app code after I upgraded stone so I expect that method to have been recompiled. In fact, I just edited the method and saved it again (so that a recompilation was triggered) and the above closure still appears as needing compilation!

Even worst....   the _method  instVar of that example IS NOT IDENTICAL (same OOP) to the one installed in the method dictionary (FaDBSessionsByRolesFactory >> #sessionMapFor:prepareAccessLimitingSessionsWith: ). So it looks like if they were un-installed closures. 

I thought it could be some MC version history around or whatever. So I performed my typical GC and cleanup code which should collect all that garbage. Still same numbers.

mmmm I am running out of ideas. 

Thoughts?
I imagine that the #sessionMapFor:prepareAccessLimitingSessionsWith: method is in an older version of the class, which would explain the differences in oops and the fact that when you recompile the method in the current version of the class, you don't eliminate block that needs to be recompiled ...

`GsDeployer new classesWithClassHistory` should give you the list of classes that have older versions ...

Dale

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Migrating from 3.2.9 to 3.3.3 block compilation issue

GLASS mailing list


On Wed, Feb 1, 2017 at 2:20 AM, Dale Henrichs <[hidden email]> wrote:
On 1/31/17 7:44 PM, Mariano Martinez Peck wrote:


On Tue, Jan 31, 2017 at 2:25 PM, Dale Henrichs via Glass <[hidden email]> wrote:

Mariano,

1. Find allInstances of ExecBlock

2. Select all instances that #needsRecompile.

3. Send #recompile to the selected instances and collect instances that signal an error during #recompile. The #recompile method attempts to recompile Simple blocks in-place if there is an error during compilation or the block is not simple, an Error is signaled.

4. For the remaining instances of ExecBlock, you will have to track down the reference to the block and reinstall a new, recompiled instance of the ExecBlock.

    1. look at the #blockSource and the #receiver for the block and you might recognize the method needed to recreate the block

    2. look at the #blockSource and pick out a likely chunk of source that you can use to search in method source (`browse source --exact `<source-chunk>`)

    3. Use Repository>>findReferencePathToObject: or Repository>>findAllReferencePathsToObject: to find which object(s) are holding onto the block.


Hi Dale,

OK, it seems I am not having good luck. I tried the following code:

[
| allBlocks allBlocksNeedingRecompile blocksFailedToRecompile correctlyRecompiled | 
blocksFailedToRecompile := OrderedCollection new. 
correctlyRecompiled := OrderedCollection new. 
System commit. 
allBlocks := ExecBlock allSubInstances.
allBlocksNeedingRecompile := allBlocks select: [:each | each needsRecompile ]. 
allBlocksNeedingRecompile do: [:each |
each recompile.
correctlyRecompiled add: each.
  ] on: Error do: [:ex | 
blocksFailedToRecompile add: each.
]
].
System commit. 
Transcript show:  'Correctly recompiled closures: ',  correctlyRecompiled size asString.
Transcript show: 'Erred recompiled closures: ',  blocksFailedToRecompile size asString.
] on: Warning do: [:ex | ex resume  ]


And it outputs:

Correctly recompiled closures: 0Erred recompiled closures: 797155

Wow. That's a lot to fix. 

But things get strange when I analyze which are the failing closures. For example,  I have an ExecBlock whose source is:

"This is source for a block.  " 
 ^ [ 
     self error:
         'No concrete db roles associated to '''
           , (FaDBAccessor common users row: username column: 'CONCRETE ROLE')
           , ''' of user ''' , username
           ,
             '''. Either the concrete db role id is wrong or the roles must be added to concreteRoles.csv' 
]



That closure, is a closure of an ifEmpty:  of the method #sessionMapFor:prepareAccessLimitingSessionsWith:
See:

sessionMapFor: username prepareAccessLimitingSessionsWith: aDictionaryOfPreparationBlocks
| answer  concreteRole abstractRoleID  |
answer := IdentityDictionary new.
concreteRole := self concreteRoleForUsername: username. 
concreteRole
   ifEmpty: [ 
     self error:
         'No concrete db roles associated to '''
           , (FaDBAccessor common users row: username column: 'CONCRETE ROLE')
           , ''' of user ''' , username
           ,
             '''. Either the concrete db role id is wrong or the roles must be added to concreteRoles.csv' 
].
.....


What is going on? I re-load all my app code after I upgraded stone so I expect that method to have been recompiled. In fact, I just edited the method and saved it again (so that a recompilation was triggered) and the above closure still appears as needing compilation!

Even worst....   the _method  instVar of that example IS NOT IDENTICAL (same OOP) to the one installed in the method dictionary (FaDBSessionsByRolesFactory >> #sessionMapFor:prepareAccessLimitingSessionsWith: ). So it looks like if they were un-installed closures. 

I thought it could be some MC version history around or whatever. So I performed my typical GC and cleanup code which should collect all that garbage. Still same numbers.

mmmm I am running out of ideas. 

Thoughts?
I imagine that the #sessionMapFor:prepareAccessLimitingSessionsWith: method is in an older version of the class, which would explain the differences in oops and the fact that when you recompile the method in the current version of the class, you don't eliminate block that needs to be recompiled ...

`GsDeployer new classesWithClassHistory` should give you the list of classes that have older versions ...

Hi Dale,

I tried with that and it answers an empty array. 
 
Now, I tried in a much smaller stone (also migrated from 3.2.9 to 3.3.3) and I also see quite some:

Correctly recompiled closures: 4917. Erred recompiled closures: 30807

And I still see the same scenario.. closures that come from ifEmpty or ifAbsent or things like that that should be gone when method is recompiled... Some of these closures/methods are not even from my app but from tODE. 

With some code to fix the closures I faced while running some parts of the app, I started to make the app to work. I cannot spend much longer testing the general issue but instead try to make the app to work. It's bad because I may lazily find more and more closures needing to recompile as the app receives more testing. 

Anyway, I will do some more tests with findAllReferencePathsToObject: to see if I found the issue...

In case you think there could be something you may want to try / do let me know (compressed backup of this extent is only 80MB).

Thanks for your help!


--

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Migrating from 3.2.9 to 3.3.3 block compilation issue

GLASS mailing list



On 02/01/2017 04:40 AM, Mariano Martinez Peck wrote:


On Wed, Feb 1, 2017 at 2:20 AM, Dale Henrichs <[hidden email]> wrote:
On 1/31/17 7:44 PM, Mariano Martinez Peck wrote:


On Tue, Jan 31, 2017 at 2:25 PM, Dale Henrichs via Glass <[hidden email]> wrote:

Mariano,

1. Find allInstances of ExecBlock

2. Select all instances that #needsRecompile.

3. Send #recompile to the selected instances and collect instances that signal an error during #recompile. The #recompile method attempts to recompile Simple blocks in-place if there is an error during compilation or the block is not simple, an Error is signaled.

4. For the remaining instances of ExecBlock, you will have to track down the reference to the block and reinstall a new, recompiled instance of the ExecBlock.

    1. look at the #blockSource and the #receiver for the block and you might recognize the method needed to recreate the block

    2. look at the #blockSource and pick out a likely chunk of source that you can use to search in method source (`browse source --exact `<source-chunk>`)

    3. Use Repository>>findReferencePathToObject: or Repository>>findAllReferencePathsToObject: to find which object(s) are holding onto the block.


Hi Dale,

OK, it seems I am not having good luck. I tried the following code:

[
| allBlocks allBlocksNeedingRecompile blocksFailedToRecompile correctlyRecompiled | 
blocksFailedToRecompile := OrderedCollection new. 
correctlyRecompiled := OrderedCollection new. 
System commit. 
allBlocks := ExecBlock allSubInstances.
allBlocksNeedingRecompile := allBlocks select: [:each | each needsRecompile ]. 
allBlocksNeedingRecompile do: [:each |
each recompile.
correctlyRecompiled add: each.
  ] on: Error do: [:ex | 
blocksFailedToRecompile add: each.
]
].
System commit. 
Transcript show:  'Correctly recompiled closures: ',  correctlyRecompiled size asString.
Transcript show: 'Erred recompiled closures: ',  blocksFailedToRecompile size asString.
] on: Warning do: [:ex | ex resume  ]


And it outputs:

Correctly recompiled closures: 0Erred recompiled closures: 797155

Wow. That's a lot to fix. 

But things get strange when I analyze which are the failing closures. For example,  I have an ExecBlock whose source is:

"This is source for a block.  " 
 ^ [ 
     self error:
         'No concrete db roles associated to '''
           , (FaDBAccessor common users row: username column: 'CONCRETE ROLE')
           , ''' of user ''' , username
           ,
             '''. Either the concrete db role id is wrong or the roles must be added to concreteRoles.csv' 
]



That closure, is a closure of an ifEmpty:  of the method #sessionMapFor:prepareAccessLimitingSessionsWith:
See:

sessionMapFor: username prepareAccessLimitingSessionsWith: aDictionaryOfPreparationBlocks
| answer  concreteRole abstractRoleID  |
answer := IdentityDictionary new.
concreteRole := self concreteRoleForUsername: username. 
concreteRole
   ifEmpty: [ 
     self error:
         'No concrete db roles associated to '''
           , (FaDBAccessor common users row: username column: 'CONCRETE ROLE')
           , ''' of user ''' , username
           ,
             '''. Either the concrete db role id is wrong or the roles must be added to concreteRoles.csv' 
].
.....


What is going on? I re-load all my app code after I upgraded stone so I expect that method to have been recompiled. In fact, I just edited the method and saved it again (so that a recompilation was triggered) and the above closure still appears as needing compilation!

Even worst....   the _method  instVar of that example IS NOT IDENTICAL (same OOP) to the one installed in the method dictionary (FaDBSessionsByRolesFactory >> #sessionMapFor:prepareAccessLimitingSessionsWith: ). So it looks like if they were un-installed closures. 

I thought it could be some MC version history around or whatever. So I performed my typical GC and cleanup code which should collect all that garbage. Still same numbers.

mmmm I am running out of ideas. 

Thoughts?
I imagine that the #sessionMapFor:prepareAccessLimitingSessionsWith: method is in an older version of the class, which would explain the differences in oops and the fact that when you recompile the method in the current version of the class, you don't eliminate block that needs to be recompiled ...

`GsDeployer new classesWithClassHistory` should give you the list of classes that have older versions ...

Hi Dale,

I tried with that and it answers an empty array. 
 
Now, I tried in a much smaller stone (also migrated from 3.2.9 to 3.3.3) and I also see quite some:

Correctly recompiled closures: 4917. Erred recompiled closures: 30807

And I still see the same scenario.. closures that come from ifEmpty or ifAbsent or things like that that should be gone when method is recompiled... Some of these closures/methods are not even from my app but from tODE. 

With some code to fix the closures I faced while running some parts of the app, I started to make the app to work. I cannot spend much longer testing the general issue but instead try to make the app to work. It's bad because I may lazily find more and more closures needing to recompile as the app receives more testing. 

Anyway, I will do some more tests with findAllReferencePathsToObject: to see if I found the issue...

In case you think there could be something you may want to try / do let me know (compressed backup of this extent is only 80MB).

Thanks for your help!

I assume that you haven't  done an MFC after the upgrade. At this point it seems likely that you are looking at the old versions of the methods...

This is probably an important step to add to the instructions ... step 0: run an mfc ...

Dale



_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Migrating from 3.2.9 to 3.3.3 block compilation issue

GLASS mailing list


On Wed, Feb 1, 2017 at 1:20 PM, Dale Henrichs <[hidden email]> wrote:



On 02/01/2017 04:40 AM, Mariano Martinez Peck wrote:


On Wed, Feb 1, 2017 at 2:20 AM, Dale Henrichs <[hidden email]> wrote:
On 1/31/17 7:44 PM, Mariano Martinez Peck wrote:


On Tue, Jan 31, 2017 at 2:25 PM, Dale Henrichs via Glass <[hidden email]> wrote:

Mariano,

1. Find allInstances of ExecBlock

2. Select all instances that #needsRecompile.

3. Send #recompile to the selected instances and collect instances that signal an error during #recompile. The #recompile method attempts to recompile Simple blocks in-place if there is an error during compilation or the block is not simple, an Error is signaled.

4. For the remaining instances of ExecBlock, you will have to track down the reference to the block and reinstall a new, recompiled instance of the ExecBlock.

    1. look at the #blockSource and the #receiver for the block and you might recognize the method needed to recreate the block

    2. look at the #blockSource and pick out a likely chunk of source that you can use to search in method source (`browse source --exact `<source-chunk>`)

    3. Use Repository>>findReferencePathToObject: or Repository>>findAllReferencePathsToObject: to find which object(s) are holding onto the block.


Hi Dale,

OK, it seems I am not having good luck. I tried the following code:

[
| allBlocks allBlocksNeedingRecompile blocksFailedToRecompile correctlyRecompiled | 
blocksFailedToRecompile := OrderedCollection new. 
correctlyRecompiled := OrderedCollection new. 
System commit. 
allBlocks := ExecBlock allSubInstances.
allBlocksNeedingRecompile := allBlocks select: [:each | each needsRecompile ]. 
allBlocksNeedingRecompile do: [:each |
each recompile.
correctlyRecompiled add: each.
  ] on: Error do: [:ex | 
blocksFailedToRecompile add: each.
]
].
System commit. 
Transcript show:  'Correctly recompiled closures: ',  correctlyRecompiled size asString.
Transcript show: 'Erred recompiled closures: ',  blocksFailedToRecompile size asString.
] on: Warning do: [:ex | ex resume  ]


And it outputs:

Correctly recompiled closures: 0Erred recompiled closures: 797155

Wow. That's a lot to fix. 

But things get strange when I analyze which are the failing closures. For example,  I have an ExecBlock whose source is:

"This is source for a block.  " 
 ^ [ 
     self error:
         'No concrete db roles associated to '''
           , (FaDBAccessor common users row: username column: 'CONCRETE ROLE')
           , ''' of user ''' , username
           ,
             '''. Either the concrete db role id is wrong or the roles must be added to concreteRoles.csv' 
]



That closure, is a closure of an ifEmpty:  of the method #sessionMapFor:prepareAccessLimitingSessionsWith:
See:

sessionMapFor: username prepareAccessLimitingSessionsWith: aDictionaryOfPreparationBlocks
| answer  concreteRole abstractRoleID  |
answer := IdentityDictionary new.
concreteRole := self concreteRoleForUsername: username. 
concreteRole
   ifEmpty: [ 
     self error:
         'No concrete db roles associated to '''
           , (FaDBAccessor common users row: username column: 'CONCRETE ROLE')
           , ''' of user ''' , username
           ,
             '''. Either the concrete db role id is wrong or the roles must be added to concreteRoles.csv' 
].
.....


What is going on? I re-load all my app code after I upgraded stone so I expect that method to have been recompiled. In fact, I just edited the method and saved it again (so that a recompilation was triggered) and the above closure still appears as needing compilation!

Even worst....   the _method  instVar of that example IS NOT IDENTICAL (same OOP) to the one installed in the method dictionary (FaDBSessionsByRolesFactory >> #sessionMapFor:prepareAccessLimitingSessionsWith: ). So it looks like if they were un-installed closures. 

I thought it could be some MC version history around or whatever. So I performed my typical GC and cleanup code which should collect all that garbage. Still same numbers.

mmmm I am running out of ideas. 

Thoughts?
I imagine that the #sessionMapFor:prepareAccessLimitingSessionsWith: method is in an older version of the class, which would explain the differences in oops and the fact that when you recompile the method in the current version of the class, you don't eliminate block that needs to be recompiled ...

`GsDeployer new classesWithClassHistory` should give you the list of classes that have older versions ...

Hi Dale,

I tried with that and it answers an empty array. 
 
Now, I tried in a much smaller stone (also migrated from 3.2.9 to 3.3.3) and I also see quite some:

Correctly recompiled closures: 4917. Erred recompiled closures: 30807

And I still see the same scenario.. closures that come from ifEmpty or ifAbsent or things like that that should be gone when method is recompiled... Some of these closures/methods are not even from my app but from tODE. 

With some code to fix the closures I faced while running some parts of the app, I started to make the app to work. I cannot spend much longer testing the general issue but instead try to make the app to work. It's bad because I may lazily find more and more closures needing to recompile as the app receives more testing. 

Anyway, I will do some more tests with findAllReferencePathsToObject: to see if I found the issue...

In case you think there could be something you may want to try / do let me know (compressed backup of this extent is only 80MB).

Thanks for your help!

I assume that you haven't  done an MFC after the upgrade. At this point it seems likely that you are looking at the old versions of the methods...

This is probably an important step to add to the instructions ... step 0: run an mfc ...


I did that, but I admit it took me some time to realize that could be a problem. In fact, in that very same stone, the 30k decreased to 600 (still quite some to track down!). So yeah, I think such a step 0 MFC would be nice to add to docs. I don't know how to open an issue for that.

Nevertheless, in the other stone (the 50GB one) I keep having a bit less than a million closures even after MFC.  



--

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Migrating from 3.2.9 to 3.3.3 block compilation issue

GLASS mailing list



On 02/01/2017 09:09 AM, Mariano Martinez Peck wrote:


On Wed, Feb 1, 2017 at 1:20 PM, Dale Henrichs <[hidden email]> wrote:



On 02/01/2017 04:40 AM, Mariano Martinez Peck wrote:


On Wed, Feb 1, 2017 at 2:20 AM, Dale Henrichs <[hidden email]> wrote:
On 1/31/17 7:44 PM, Mariano Martinez Peck wrote:


On Tue, Jan 31, 2017 at 2:25 PM, Dale Henrichs via Glass <[hidden email]> wrote:

Mariano,

1. Find allInstances of ExecBlock

2. Select all instances that #needsRecompile.

3. Send #recompile to the selected instances and collect instances that signal an error during #recompile. The #recompile method attempts to recompile Simple blocks in-place if there is an error during compilation or the block is not simple, an Error is signaled.

4. For the remaining instances of ExecBlock, you will have to track down the reference to the block and reinstall a new, recompiled instance of the ExecBlock.

    1. look at the #blockSource and the #receiver for the block and you might recognize the method needed to recreate the block

    2. look at the #blockSource and pick out a likely chunk of source that you can use to search in method source (`browse source --exact `<source-chunk>`)

    3. Use Repository>>findReferencePathToObject: or Repository>>findAllReferencePathsToObject: to find which object(s) are holding onto the block.


Hi Dale,

OK, it seems I am not having good luck. I tried the following code:

[
| allBlocks allBlocksNeedingRecompile blocksFailedToRecompile correctlyRecompiled | 
blocksFailedToRecompile := OrderedCollection new. 
correctlyRecompiled := OrderedCollection new. 
System commit. 
allBlocks := ExecBlock allSubInstances.
allBlocksNeedingRecompile := allBlocks select: [:each | each needsRecompile ]. 
allBlocksNeedingRecompile do: [:each |
each recompile.
correctlyRecompiled add: each.
  ] on: Error do: [:ex | 
blocksFailedToRecompile add: each.
]
].
System commit. 
Transcript show:  'Correctly recompiled closures: ',  correctlyRecompiled size asString.
Transcript show: 'Erred recompiled closures: ',  blocksFailedToRecompile size asString.
] on: Warning do: [:ex | ex resume  ]


And it outputs:

Correctly recompiled closures: 0Erred recompiled closures: 797155

Wow. That's a lot to fix. 

But things get strange when I analyze which are the failing closures. For example,  I have an ExecBlock whose source is:

"This is source for a block.  " 
 ^ [ 
     self error:
         'No concrete db roles associated to '''
           , (FaDBAccessor common users row: username column: 'CONCRETE ROLE')
           , ''' of user ''' , username
           ,
             '''. Either the concrete db role id is wrong or the roles must be added to concreteRoles.csv' 
]



That closure, is a closure of an ifEmpty:  of the method #sessionMapFor:prepareAccessLimitingSessionsWith:
See:

sessionMapFor: username prepareAccessLimitingSessionsWith: aDictionaryOfPreparationBlocks
| answer  concreteRole abstractRoleID  |
answer := IdentityDictionary new.
concreteRole := self concreteRoleForUsername: username. 
concreteRole
   ifEmpty: [ 
     self error:
         'No concrete db roles associated to '''
           , (FaDBAccessor common users row: username column: 'CONCRETE ROLE')
           , ''' of user ''' , username
           ,
             '''. Either the concrete db role id is wrong or the roles must be added to concreteRoles.csv' 
].
.....


What is going on? I re-load all my app code after I upgraded stone so I expect that method to have been recompiled. In fact, I just edited the method and saved it again (so that a recompilation was triggered) and the above closure still appears as needing compilation!

Even worst....   the _method  instVar of that example IS NOT IDENTICAL (same OOP) to the one installed in the method dictionary (FaDBSessionsByRolesFactory >> #sessionMapFor:prepareAccessLimitingSessionsWith: ). So it looks like if they were un-installed closures. 

I thought it could be some MC version history around or whatever. So I performed my typical GC and cleanup code which should collect all that garbage. Still same numbers.

mmmm I am running out of ideas. 

Thoughts?
I imagine that the #sessionMapFor:prepareAccessLimitingSessionsWith: method is in an older version of the class, which would explain the differences in oops and the fact that when you recompile the method in the current version of the class, you don't eliminate block that needs to be recompiled ...

`GsDeployer new classesWithClassHistory` should give you the list of classes that have older versions ...

Hi Dale,

I tried with that and it answers an empty array. 
 
Now, I tried in a much smaller stone (also migrated from 3.2.9 to 3.3.3) and I also see quite some:

Correctly recompiled closures: 4917. Erred recompiled closures: 30807

And I still see the same scenario.. closures that come from ifEmpty or ifAbsent or things like that that should be gone when method is recompiled... Some of these closures/methods are not even from my app but from tODE. 

With some code to fix the closures I faced while running some parts of the app, I started to make the app to work. I cannot spend much longer testing the general issue but instead try to make the app to work. It's bad because I may lazily find more and more closures needing to recompile as the app receives more testing. 

Anyway, I will do some more tests with findAllReferencePathsToObject: to see if I found the issue...

In case you think there could be something you may want to try / do let me know (compressed backup of this extent is only 80MB).

Thanks for your help!

I assume that you haven't  done an MFC after the upgrade. At this point it seems likely that you are looking at the old versions of the methods...

This is probably an important step to add to the instructions ... step 0: run an mfc ...


I did that, but I admit it took me some time to realize that could be a problem. In fact, in that very same stone, the 30k decreased to 600 (still quite some to track down!). So yeah, I think such a step 0 MFC would be nice to add to docs. I don't know how to open an issue for that.
internal bug #46657 (I already opened one:)

Nevertheless, in the other stone (the 50GB one) I keep having a bit less than a million closures even after MFC.  


I think we need to become a bit more methodical in tracking this down ...  if you've done an all instances run, you should avoid persisting the collection and you should probably logout of the session before doing an MFC.

So I would like to know that there are no classes in class history ...

Given an MFC and no classes in class history, I would be surprised to see blocks that are of the #isEmpty style .. if you have these types of blocks still alive, I would think that we'd need to do an a reference path analysis on them ... I would hope that the methods that are returned and held onto in random spots would be recognizable and that the reference path could be tracked down by looking senders of the method where the block was created ...

It just occurred to me that you probably should clear the object log or at a minimum delete all of the continuations in the object log, since they would be another place where old methods would be kept alive ...

Dale


_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Migrating from 3.2.9 to 3.3.3 block compilation issue

GLASS mailing list


On Wed, Feb 1, 2017 at 2:24 PM, Dale Henrichs <[hidden email]> wrote:



On 02/01/2017 09:09 AM, Mariano Martinez Peck wrote:


On Wed, Feb 1, 2017 at 1:20 PM, Dale Henrichs <[hidden email]> wrote:



On 02/01/2017 04:40 AM, Mariano Martinez Peck wrote:


On Wed, Feb 1, 2017 at 2:20 AM, Dale Henrichs <[hidden email]> wrote:
On 1/31/17 7:44 PM, Mariano Martinez Peck wrote:


On Tue, Jan 31, 2017 at 2:25 PM, Dale Henrichs via Glass <[hidden email]> wrote:

Mariano,

1. Find allInstances of ExecBlock

2. Select all instances that #needsRecompile.

3. Send #recompile to the selected instances and collect instances that signal an error during #recompile. The #recompile method attempts to recompile Simple blocks in-place if there is an error during compilation or the block is not simple, an Error is signaled.

4. For the remaining instances of ExecBlock, you will have to track down the reference to the block and reinstall a new, recompiled instance of the ExecBlock.

    1. look at the #blockSource and the #receiver for the block and you might recognize the method needed to recreate the block

    2. look at the #blockSource and pick out a likely chunk of source that you can use to search in method source (`browse source --exact `<source-chunk>`)

    3. Use Repository>>findReferencePathToObject: or Repository>>findAllReferencePathsToObject: to find which object(s) are holding onto the block.


Hi Dale,

OK, it seems I am not having good luck. I tried the following code:

[
| allBlocks allBlocksNeedingRecompile blocksFailedToRecompile correctlyRecompiled | 
blocksFailedToRecompile := OrderedCollection new. 
correctlyRecompiled := OrderedCollection new. 
System commit. 
allBlocks := ExecBlock allSubInstances.
allBlocksNeedingRecompile := allBlocks select: [:each | each needsRecompile ]. 
allBlocksNeedingRecompile do: [:each |
each recompile.
correctlyRecompiled add: each.
  ] on: Error do: [:ex | 
blocksFailedToRecompile add: each.
]
].
System commit. 
Transcript show:  'Correctly recompiled closures: ',  correctlyRecompiled size asString.
Transcript show: 'Erred recompiled closures: ',  blocksFailedToRecompile size asString.
] on: Warning do: [:ex | ex resume  ]


And it outputs:

Correctly recompiled closures: 0Erred recompiled closures: 797155

Wow. That's a lot to fix. 

But things get strange when I analyze which are the failing closures. For example,  I have an ExecBlock whose source is:

"This is source for a block.  " 
 ^ [ 
     self error:
         'No concrete db roles associated to '''
           , (FaDBAccessor common users row: username column: 'CONCRETE ROLE')
           , ''' of user ''' , username
           ,
             '''. Either the concrete db role id is wrong or the roles must be added to concreteRoles.csv' 
]



That closure, is a closure of an ifEmpty:  of the method #sessionMapFor:prepareAccessLimitingSessionsWith:
See:

sessionMapFor: username prepareAccessLimitingSessionsWith: aDictionaryOfPreparationBlocks
| answer  concreteRole abstractRoleID  |
answer := IdentityDictionary new.
concreteRole := self concreteRoleForUsername: username. 
concreteRole
   ifEmpty: [ 
     self error:
         'No concrete db roles associated to '''
           , (FaDBAccessor common users row: username column: 'CONCRETE ROLE')
           , ''' of user ''' , username
           ,
             '''. Either the concrete db role id is wrong or the roles must be added to concreteRoles.csv' 
].
.....


What is going on? I re-load all my app code after I upgraded stone so I expect that method to have been recompiled. In fact, I just edited the method and saved it again (so that a recompilation was triggered) and the above closure still appears as needing compilation!

Even worst....   the _method  instVar of that example IS NOT IDENTICAL (same OOP) to the one installed in the method dictionary (FaDBSessionsByRolesFactory >> #sessionMapFor:prepareAccessLimitingSessionsWith: ). So it looks like if they were un-installed closures. 

I thought it could be some MC version history around or whatever. So I performed my typical GC and cleanup code which should collect all that garbage. Still same numbers.

mmmm I am running out of ideas. 

Thoughts?
I imagine that the #sessionMapFor:prepareAccessLimitingSessionsWith: method is in an older version of the class, which would explain the differences in oops and the fact that when you recompile the method in the current version of the class, you don't eliminate block that needs to be recompiled ...

`GsDeployer new classesWithClassHistory` should give you the list of classes that have older versions ...

Hi Dale,

I tried with that and it answers an empty array. 
 
Now, I tried in a much smaller stone (also migrated from 3.2.9 to 3.3.3) and I also see quite some:

Correctly recompiled closures: 4917. Erred recompiled closures: 30807

And I still see the same scenario.. closures that come from ifEmpty or ifAbsent or things like that that should be gone when method is recompiled... Some of these closures/methods are not even from my app but from tODE. 

With some code to fix the closures I faced while running some parts of the app, I started to make the app to work. I cannot spend much longer testing the general issue but instead try to make the app to work. It's bad because I may lazily find more and more closures needing to recompile as the app receives more testing. 

Anyway, I will do some more tests with findAllReferencePathsToObject: to see if I found the issue...

In case you think there could be something you may want to try / do let me know (compressed backup of this extent is only 80MB).

Thanks for your help!

I assume that you haven't  done an MFC after the upgrade. At this point it seems likely that you are looking at the old versions of the methods...

This is probably an important step to add to the instructions ... step 0: run an mfc ...


I did that, but I admit it took me some time to realize that could be a problem. In fact, in that very same stone, the 30k decreased to 600 (still quite some to track down!). So yeah, I think such a step 0 MFC would be nice to add to docs. I don't know how to open an issue for that.
internal bug #46657 (I already opened one:)

Nevertheless, in the other stone (the 50GB one) I keep having a bit less than a million closures even after MFC.  


I think we need to become a bit more methodical in tracking this down ...  if you've done an all instances run, you should avoid persisting the collection and you should probably logout of the session before doing an MFC.


Yes, I do this. 
 
So I would like to know that there are no classes in class history ...


Well, here i just discovered an issue...

`GsDeployer new classesWithClassHistory`    shows empty (as I told you earlier). 

But... tODE `cls history` shows a bunch!  And in fact, the examples I found in the small stone seem to be from the classes listed in `cls history` ... 

I have to admit that I am not sure I like #classesWithClassHistory to NOT show me certain classes even if they are "core" from GemSTone. I pretty much have them all (like `cls history`) and decide myself which ones are important to me. 

 
Given an MFC and no classes in class history, I would be surprised to see blocks that are of the #isEmpty style .. if you have these types of blocks still alive, I would think that we'd need to do an a reference path analysis on them ... I would hope that the methods that are returned and held onto in random spots would be recognizable and that the reference path could be tracked down by looking senders of the method where the block was created ...

It just occurred to me that you probably should clear the object log or at a minimum delete all of the continuations in the object log, since they would be another place where old methods would be kept alive ...


Yes, my cleaning also purges the object log completly. 

So..as for the "small " stone it looks like I get down to do those from different class versions of GemStone classes. So it seems good enough. 

As for the big stone (50GB) I am still having a million closures and many are NOT from class history > 1 .. .so I am still investigating this one.

Cheers.






--

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Migrating from 3.2.9 to 3.3.3 block compilation issue

GLASS mailing list



On 02/01/2017 11:33 AM, Mariano Martinez Peck wrote:


On Wed, Feb 1, 2017 at 2:24 PM, Dale Henrichs <[hidden email]> wrote:



On 02/01/2017 09:09 AM, Mariano Martinez Peck wrote:


On Wed, Feb 1, 2017 at 1:20 PM, Dale Henrichs <[hidden email]> wrote:



On 02/01/2017 04:40 AM, Mariano Martinez Peck wrote:


On Wed, Feb 1, 2017 at 2:20 AM, Dale Henrichs <[hidden email]> wrote:
On 1/31/17 7:44 PM, Mariano Martinez Peck wrote:


On Tue, Jan 31, 2017 at 2:25 PM, Dale Henrichs via Glass <[hidden email]> wrote:

Mariano,

1. Find allInstances of ExecBlock

2. Select all instances that #needsRecompile.

3. Send #recompile to the selected instances and collect instances that signal an error during #recompile. The #recompile method attempts to recompile Simple blocks in-place if there is an error during compilation or the block is not simple, an Error is signaled.

4. For the remaining instances of ExecBlock, you will have to track down the reference to the block and reinstall a new, recompiled instance of the ExecBlock.

    1. look at the #blockSource and the #receiver for the block and you might recognize the method needed to recreate the block

    2. look at the #blockSource and pick out a likely chunk of source that you can use to search in method source (`browse source --exact `<source-chunk>`)

    3. Use Repository>>findReferencePathToObject: or Repository>>findAllReferencePathsToObject: to find which object(s) are holding onto the block.


Hi Dale,

OK, it seems I am not having good luck. I tried the following code:

[
| allBlocks allBlocksNeedingRecompile blocksFailedToRecompile correctlyRecompiled | 
blocksFailedToRecompile := OrderedCollection new. 
correctlyRecompiled := OrderedCollection new. 
System commit. 
allBlocks := ExecBlock allSubInstances.
allBlocksNeedingRecompile := allBlocks select: [:each | each needsRecompile ]. 
allBlocksNeedingRecompile do: [:each |
each recompile.
correctlyRecompiled add: each.
  ] on: Error do: [:ex | 
blocksFailedToRecompile add: each.
]
].
System commit. 
Transcript show:  'Correctly recompiled closures: ',  correctlyRecompiled size asString.
Transcript show: 'Erred recompiled closures: ',  blocksFailedToRecompile size asString.
] on: Warning do: [:ex | ex resume  ]


And it outputs:

Correctly recompiled closures: 0Erred recompiled closures: 797155

Wow. That's a lot to fix. 

But things get strange when I analyze which are the failing closures. For example,  I have an ExecBlock whose source is:

"This is source for a block.  " 
 ^ [ 
     self error:
         'No concrete db roles associated to '''
           , (FaDBAccessor common users row: username column: 'CONCRETE ROLE')
           , ''' of user ''' , username
           ,
             '''. Either the concrete db role id is wrong or the roles must be added to concreteRoles.csv' 
]



That closure, is a closure of an ifEmpty:  of the method #sessionMapFor:prepareAccessLimitingSessionsWith:
See:

sessionMapFor: username prepareAccessLimitingSessionsWith: aDictionaryOfPreparationBlocks
| answer  concreteRole abstractRoleID  |
answer := IdentityDictionary new.
concreteRole := self concreteRoleForUsername: username. 
concreteRole
   ifEmpty: [ 
     self error:
         'No concrete db roles associated to '''
           , (FaDBAccessor common users row: username column: 'CONCRETE ROLE')
           , ''' of user ''' , username
           ,
             '''. Either the concrete db role id is wrong or the roles must be added to concreteRoles.csv' 
].
.....


What is going on? I re-load all my app code after I upgraded stone so I expect that method to have been recompiled. In fact, I just edited the method and saved it again (so that a recompilation was triggered) and the above closure still appears as needing compilation!

Even worst....   the _method  instVar of that example IS NOT IDENTICAL (same OOP) to the one installed in the method dictionary (FaDBSessionsByRolesFactory >> #sessionMapFor:prepareAccessLimitingSessionsWith: ). So it looks like if they were un-installed closures. 

I thought it could be some MC version history around or whatever. So I performed my typical GC and cleanup code which should collect all that garbage. Still same numbers.

mmmm I am running out of ideas. 

Thoughts?
I imagine that the #sessionMapFor:prepareAccessLimitingSessionsWith: method is in an older version of the class, which would explain the differences in oops and the fact that when you recompile the method in the current version of the class, you don't eliminate block that needs to be recompiled ...

`GsDeployer new classesWithClassHistory` should give you the list of classes that have older versions ...

Hi Dale,

I tried with that and it answers an empty array. 
 
Now, I tried in a much smaller stone (also migrated from 3.2.9 to 3.3.3) and I also see quite some:

Correctly recompiled closures: 4917. Erred recompiled closures: 30807

And I still see the same scenario.. closures that come from ifEmpty or ifAbsent or things like that that should be gone when method is recompiled... Some of these closures/methods are not even from my app but from tODE. 

With some code to fix the closures I faced while running some parts of the app, I started to make the app to work. I cannot spend much longer testing the general issue but instead try to make the app to work. It's bad because I may lazily find more and more closures needing to recompile as the app receives more testing. 

Anyway, I will do some more tests with findAllReferencePathsToObject: to see if I found the issue...

In case you think there could be something you may want to try / do let me know (compressed backup of this extent is only 80MB).

Thanks for your help!

I assume that you haven't  done an MFC after the upgrade. At this point it seems likely that you are looking at the old versions of the methods...

This is probably an important step to add to the instructions ... step 0: run an mfc ...


I did that, but I admit it took me some time to realize that could be a problem. In fact, in that very same stone, the 30k decreased to 600 (still quite some to track down!). So yeah, I think such a step 0 MFC would be nice to add to docs. I don't know how to open an issue for that.
internal bug #46657 (I already opened one:)

Nevertheless, in the other stone (the 50GB one) I keep having a bit less than a million closures even after MFC.  


I think we need to become a bit more methodical in tracking this down ...  if you've done an all instances run, you should avoid persisting the collection and you should probably logout of the session before doing an MFC.


Yes, I do this. 
 
So I would like to know that there are no classes in class history ...


Well, here i just discovered an issue...

`GsDeployer new classesWithClassHistory`    shows empty (as I told you earlier). 

But... tODE `cls history` shows a bunch!  And in fact, the examples I found in the small stone seem to be from the classes listed in `cls history` ... 

I have to admit that I am not sure I like #classesWithClassHistory to NOT show me certain classes even if they are "core" from GemSTone. I pretty much have them all (like `cls history`) and decide myself which ones are important to me.

I had forgotten that GsDeployer had the excluded Globals list ... sorry about that ...

Dale

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Loading...