Memory leaks

Hi!

It looks like that we are currently suffering memory leaks.
In march 2015 it was said in the mailing list:

-=-=-=-=-=-=-=-=
I also tried this:

1. Create a dummy class:
Object subclass: #AAA
        instanceVariableNames: 'x'
        classVariableNames: ''
        category: 'AAA'.
       
2. Open a Playground and do it and go on this:
a := #AAA asClass new.

3. In the second pane go on "self"

4. Close the Playground

5. Execute:
3 timesRepeat: [ Smalltalk garbageCollect ].

6. Check:
#AAA asClass allInstances.

7. Repeat if necessary, and after a while, the result after step 6 is not empty.
-=-=-=-=-=-=-=-=

The problem is still present.
Looking at the pointers

There was an object reference crawler no? I remember someone worked on this? I tried sending #pointersTo but I without much success on identifying the cause of this leak.

Any idea?

Cheers,
Alexandre
--
_,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:
Alexandre Bergel  http://www.bergel.eu
^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.

> On 30 Jan 2017, at 22:57, Alexandre Bergel <[hidden email]> wrote:
>
> Hi!
>
> It looks like that we are currently suffering memory leaks.

That is a bit strong a statement.
Maybe some IDE UI stuff might sometimes hold onto something too long.

And why would you do that ?

You assign to a workspace variable, hence your object is held in the scope of the Playground.

So whenever the Playground gets retained by some other object, like an announcement, event or whatever, your object get retained too.

I am 100% sure that if you do

1e6 timesRepeat: [ #AAA asClass new ].
3 timesRepeat: [ Smalltalk garbageCollect ].
#AAA asClass allInstances isEmpty.

=> true
 

I don’t know if there is an existing object reference crawler, but I ported mine from VW. It finds the shortest paths so it is good for finding the reference that is causing the memory leak.

MCHttpRepository
        location: 'http://www.smalltalkhub.com/mc/JohnBrant/ReferenceFinder/main'
        user: ''
        password: ‘’

Once loaded, you can evaluate something like "ReferenceFinder findPathToInstanceOf: MyClass”.  That will return a reference path or nil if no path exists from the Smalltalk global to an instance of MyClass. If you want to find a path to a particular instance, you can use “ReferenceFinder findPathTo: myInst”.

In the past, I’ve found that NECController and some compiler infrastructure (I forget the class) have been culprits for memory leaks with their caches.


John Brant

Thanks John

For the record we are investigating a leak on points.
In pharo 60 we discovered that in certain images I could have 300 000 or  
more points. :)
Now marcus was analysing the problems with pavel and we had the impression  
that
some of the points may have been tenured too fast.

Pavel shows that every 5 ms we get a new point.

Stef


On Tue, 31 Jan 2017 05:00:47 +0100, John Brant  
<[hidden email]> wrote:


--
Using Opera's mail client: http://www.opera.com/mail/

Hi John,

thanks for that tool. Have found a leak that I have been chasing like forever... I would vote to integrate that into the base image.

Regards,

Thierry

2017-01-31 5:00 GMT+01:00 John Brant <[hidden email]>:

> On Jan 30, 2017, at 3:57 PM, Alexandre Bergel <[hidden email]> wrote:
>
>
> The problem is still present.
> Looking at the pointers
>
> There was an object reference crawler no? I remember someone worked on this? I tried sending #pointersTo but I without much success on identifying the cause of this leak.
>
> Any idea?
>

I don’t know if there is an existing object reference crawler, but I ported mine from VW. It finds the shortest paths so it is good for finding the reference that is causing the memory leak.

MCHttpRepository
        location: 'http://www.smalltalkhub.com/mc/JohnBrant/ReferenceFinder/main'
        user: ''
        password: ‘’

Once loaded, you can evaluate something like "ReferenceFinder findPathToInstanceOf: MyClass”.  That will return a reference path or nil if no path exists from the Smalltalk global to an instance of MyClass. If you want to find a path to a particular instance, you can use “ReferenceFinder findPathTo: myInst”.

In the past, I’ve found that NECController and some compiler infrastructure (I forget the class) have been culprits for memory leaks with their caches.


John Brant

On 01/31/2017 04:02 AM, Pavel Krivanek wrote:
> Very nice tool.
> I think it cannot find objects pointed only by a stack of some processes
> so it may be handy to put them to the global space.
>
> Smalltalk globals at: #Stacks put: ((Process allInstances collect: [:p |
> (p suspendedContext) ifNotNil: #stack ]) reject: #isNil)
> asOrderedCollection.
> (Smalltalk globals at: #Stacks) add: thisContext stack.

You could change the default "from:" object from Smalltalk to the GC
root object array. Or, you could create your own object (e.g., Array
with: Smalltalk with: ...). The tool finds paths from one object to
another and doesn't care what the start object is. If the from: object
isn't specified, it defaults to Smalltalk.

As for processes, I think it will find items from suspended processes,
as they will waiting in the process scheduler or on some semaphore which
will have references. And, the tool will be the current active process
so hopefully it isn't causing the memory leak. :)


John Brant

On 01/31/2017 06:45 AM, stepharong wrote:
> thierry
>
> we have pointersTo and I imagine that it is not good enough.
> Do you confirm?

#pointersTo appears to be the same as #allReferences and #allOwners in
other Smalltalk implementations. These methods are terrible for finding
the reason for the memory leak. To find the root of the problem, you
start with some object you think should have been GCed. The #pointersTo
gives you which objects reference that object. Then you iterate picking
some object in the previous pointersTo list and asking for its
pointersTo until you get to the class variable or class instance
variable that is the real root of the problem. The difficulty with this
process is "picking some object" from the pointersTo list to do another
pointersTo. You may pick an object that is part of the inspectors that
you are using. Another common problem is picking an object that you have
already seen (cycles). I don't know about the Pharo implementation, but
some of the other implementations also have a problem of having
implementation objects showing in the pointersTo list (i.e., objects
that were created by the pointersTo process).

As for the ease of use, I have an image where I just closed my work
window, and see that I still have some PBVariableNodes laying around. If
I execute "PBVariableNode allInstances first pointersTo" I get an array
with 2 items. The first is an array with 4 items and the second is an
array with 11587 items. Now, I would need to pick which of these I would
need to do pointersTo.

If I use my ReferenceFinder, I can do "ReferenceFinder findPathTo:
PBVariableNode allInstances first" or even "ReferenceFinder
findPathToInstanceOf: PBVariableNode". This will give me an array where
the first object is the Smalltalk system dictionary and the last object
is a PBVariableNode. In my case, I see a reference path goes through the
ASTCache. That object is holding on, somewhat indirectly, to my
anonymous class that is no longer being held by anything else.
Therefore, I concluded that either the ASTCache needs to be changed so
that it no longer caches items for anonymous classes, or I need to
change my code to remove all methods from the anonymous class before
exiting (since the ASTCache uses a weak key cache on the compiled method).

BTW, the reference path also shows that it would have been wise to pick
the array of 11587 items to do the pointersTo analysis on above.


John Brant

Hi John,

Your tool is truly a gem!

Yesterday I spent a couple of hours using #pointersTo to track some Roassal objects. But I was unsuccessful. Your tool help me immediately spot kept references. In particular, the following invocation frees all the Roassal and Athens surface objects:

Clipboard startUp: true.
GTPlayBook reset.
RubEditingState allInstances do: [ :i | i undoManager reset. ].
CairoBackendCache soleInstance flush

Should be included in the image.

Alexandre
--
_,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:
Alexandre Bergel  http://www.bergel.eu
^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.

I created a config (in original repo and MetaRepoForPharo60) so it should be available
In the morning in catalog already.

Thanks to John for the nice tool and giving me access to the repo on sthub.

Thx
T.

I cannot find ReferenceFinder in the catalog.
Any idea?

Alexandre


> On Jan 31, 2017, at 8:17 PM, Torsten Bergmann <[hidden email]> wrote:
>
> I created a config (in original repo and MetaRepoForPharo60) so it should be available
> In the morning in catalog already.
>
> Thanks to John for the nice tool and giving me access to the repo on sthub.
>
> Thx
> T.

--
_,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:
Alexandre Bergel  http://www.bergel.eu
^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.