Debug why objects get committed

On 30 Mar 2021, at 19:17, Dale Henrichs via Glass <[hidden email]> wrote:

Johan,

Could you tell me the classes of the objects that you are concerned about and/or the code that is executed that creates these objects? I would like to read the code and see if I can figure something out ... it's been quite a while since I've been in that code, so giving me a pointer to the suspicious code (and a SHA or mcz of the code you are using) I can parachute in and start poking around ...

Dale

On Mar 30 2021, at 10:00 am, Johan Brichau via Glass <[hidden email]> wrote:

Hi,

I’m busy investigating why a bunch of objects get committed to the database but I’m having some trouble finding the right tools and method to investigate that.

The task that creates the objects is ran inside (a previous version of) the ServiceVM framework [1].

This task creates a lot of temporary objects and commits the result at the end.

For some reason, these intermediate temporary objects are committed to the database and since it’s quite a large number of intermediate objects this occurs in steps of 100M per task (which is also the temp obj space for the topaz running the task).

I’m trying to find the reason the objects are committed but fail to do so.

When, from a different topaz session, I ask any of those objects ’SystemRepository findReferencePathToObject: theObject’, I’m always getting a false result and when running the MFC, the objects are indeed gc’ed.

But I want to prevent to commit that garbage.

The only reason I can think of is that the link of those objects to the persistent root was through the block that was scheduled in the ServiceVM framework. Since that block is removed once the tasks has finished executing, there is no persistent path anymore to the intermediate objects and persistent root, which would confirm why I cannot find any.

Right now, I’m just poking at the code and trying to find and break the link from the persistent root to the intermediate objects before the task commits.

Are there better ways to start investigating this? Usually, when the database grows I can find the problem via #findReferencePathToObject: but now, the committed objects are already garbage when I get to them…

Any hints appreciated ;-)

cheers

Johan

[1] https://github.com/GsDevKit/ServiceVM

_______________________________________________

Glass mailing list

[hidden email]

https://lists.gemtalksystems.com/mailman/listinfo/glass

_______________________________________________
Glass mailing list
[hidden email]
https://lists.gemtalksystems.com/mailman/listinfo/glass

On 31 Mar 2021, at 01:15, Dale Henrichs <[hidden email]> wrote:

Johan,

Below is a reply from engineering, but I think that Systen class>>writtenObjects is the only method that is available in a version that is as old as you're using ... Now  #writtenObjects  only cover persistent, modified objects, but that should give you a list of root objects to start with and you could probably do your own traversal to trace the transient objects, but since they would be your classes, you might be able to figure out enough from there ...

> Before a commit, you can execute

>   System _writtenObjects

> to get a list of the modified committed objects.

>

> If you have fix 49211, i.e. are running on 3.5.5 or 3.6.1

> (3.6.1 alpha including bec99d4c5229 from Dec 3, or upcoming release))

> you can get the closure of the modified committed objects by

>  System beginNestedTransaction .

>   closureList := System enumerateDirtyList: -1 .

>   System commit .

> as documented in comments for System class >> enumerateDirtyList:

>

> For an individual not-committed object in the closure list, you can find

> in-memory reference paths by repeated execution of

>   Object >> findReferencesInMemory

> until you have a modified committed object in the result of findReferencesInMemory.

Dale

On Mar 30 2021, at 10:28 am, Johan Brichau <[hidden email]> wrote:

Hi Dale,

I will have to factor out the code to present you with a succinct case. Let me see...

The classes are Yesplan-specific classes (but just plain subclasses of Object).

We’re implementing a memoize pattern to cache their calculation.

The instances are basically held in a bunch of IdentityDictionary instances.

Before committing the result in the service vm, I unassign the IdentityDictionary instances from the variable they are held in.

At the very least, the block that is committed in the InProcessBag is not the culprit, as I now keep it in a ‘Done’ class variable to see if I could still trace a reference path if it would still be there… but the reference path is still non existent.

Is there a way to get all the objects that would be committed before doing a commit? If I can write those out, I might find a bit more..

Johan

On 30 Mar 2021, at 19:17, Dale Henrichs via Glass <[hidden email]> wrote:

Johan,

Could you tell me the classes of the objects that you are concerned about and/or the code that is executed that creates these objects? I would like to read the code and see if I can figure something out ... it's been quite a while since I've been in that code, so giving me a pointer to the suspicious code (and a SHA or mcz of the code you are using) I can parachute in and start poking around ...

Dale

On Mar 30 2021, at 10:00 am, Johan Brichau via Glass <[hidden email]> wrote:

Hi,

I’m busy investigating why a bunch of objects get committed to the database but I’m having some trouble finding the right tools and method to investigate that.

The task that creates the objects is ran inside (a previous version of) the ServiceVM framework [1].

This task creates a lot of temporary objects and commits the result at the end.

For some reason, these intermediate temporary objects are committed to the database and since it’s quite a large number of intermediate objects this occurs in steps of 100M per task (which is also the temp obj space for the topaz running the task).

I’m trying to find the reason the objects are committed but fail to do so.

When, from a different topaz session, I ask any of those objects ’SystemRepository findReferencePathToObject: theObject’, I’m always getting a false result and when running the MFC, the objects are indeed gc’ed.

But I want to prevent to commit that garbage.

The only reason I can think of is that the link of those objects to the persistent root was through the block that was scheduled in the ServiceVM framework. Since that block is removed once the tasks has finished executing, there is no persistent path anymore to the intermediate objects and persistent root, which would confirm why I cannot find any.

Right now, I’m just poking at the code and trying to find and break the link from the persistent root to the intermediate objects before the task commits.

Are there better ways to start investigating this? Usually, when the database grows I can find the problem via #findReferencePathToObject: but now, the committed objects are already garbage when I get to them…

Any hints appreciated ;-)

cheers

Johan

[1] https://github.com/GsDevKit/ServiceVM

_______________________________________________

Glass mailing list

[hidden email]

https://lists.gemtalksystems.com/mailman/listinfo/glass

_______________________________________________

Glass mailing list

[hidden email]

https://lists.gemtalksystems.com/mailman/listinfo/glass