"Leaking" CommandLineHandler when running headless image

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

"Leaking" CommandLineHandler when running headless image

Holger Freyther
Hi,

I am currently trying to run one of my images as non-root and related to that look into keeping changes in a different directory or not writing them at all. While looking at a write failure I saw some paths being logged that I used during CI to load code and should have been GCed.


Observation:


Pharo --headless My.image eval "Smalltalk garbageCollect. CommandLineHandler allSubInstances size"
84

Pharo --headfull My.image eval "Smalltalk garbageCollect. CommandLineHandler allSubInstances size"
2


As part of the CommandLineHandler allSubInstances there are eight LoadUpdatesCommandLineHandler still active. I would assume that a full GC should have collected them by now (some of them being created in May). In a headfull image they disappear quickly

I varied the execution a bit:

Pharo --headless RoamingHub.image eval --no-quit "[ Smalltalk garbageCollect. FileStream stdout print: CommandLineHandler allSubInstances size; lf. Smalltalk snapshot: false andQuit: true] fork"
0


Hypothesis:


* Command line handlers call >>#snapshot:andQuit:.
* Image resumes in this process
* New session created
* Command line handlers execute
* Calls snapshot:andQuit:
* Image resumes..
* New session created
* Command line handlers execute
...


Can this be true? I think the proposal to start the image differently would help here? Any comments/ideas?

holger
Reply | Threaded
Open this post in threaded view
|

Re: "Leaking" CommandLineHandler when running headless image

Denis Kudriashov
Hi.

It is probably related to issue 20309

2017-09-19 15:02 GMT+02:00 Holger Freyther <[hidden email]>:
Hi,

I am currently trying to run one of my images as non-root and related to that look into keeping changes in a different directory or not writing them at all. While looking at a write failure I saw some paths being logged that I used during CI to load code and should have been GCed.


Observation:


Pharo --headless My.image eval "Smalltalk garbageCollect. CommandLineHandler allSubInstances size"
84

Pharo --headfull My.image eval "Smalltalk garbageCollect. CommandLineHandler allSubInstances size"
2


As part of the CommandLineHandler allSubInstances there are eight LoadUpdatesCommandLineHandler still active. I would assume that a full GC should have collected them by now (some of them being created in May). In a headfull image they disappear quickly

I varied the execution a bit:

Pharo --headless RoamingHub.image eval --no-quit "[ Smalltalk garbageCollect. FileStream stdout print: CommandLineHandler allSubInstances size; lf. Smalltalk snapshot: false andQuit: true] fork"
0


Hypothesis:


* Command line handlers call >>#snapshot:andQuit:.
* Image resumes in this process
* New session created
* Command line handlers execute
* Calls snapshot:andQuit:
* Image resumes..
* New session created
* Command line handlers execute
...


Can this be true? I think the proposal to start the image differently would help here? Any comments/ideas?

holger

Reply | Threaded
Open this post in threaded view
|

Re: "Leaking" CommandLineHandler when running headless image

Tim Mackinnon
There was a long post from Guille about how to sort this. It had to do with altering the priority of shutdown code - I'm not sure if we resolved it, but we should go back and see as it was well reasoned.

Tim 

Sent from my iPhone

On 19 Sep 2017, at 14:20, Denis Kudriashov <[hidden email]> wrote:

Hi.

It is probably related to issue 20309

2017-09-19 15:02 GMT+02:00 Holger Freyther <[hidden email]>:
Hi,

I am currently trying to run one of my images as non-root and related to that look into keeping changes in a different directory or not writing them at all. While looking at a write failure I saw some paths being logged that I used during CI to load code and should have been GCed.


Observation:


Pharo --headless My.image eval "Smalltalk garbageCollect. CommandLineHandler allSubInstances size"
84

Pharo --headfull My.image eval "Smalltalk garbageCollect. CommandLineHandler allSubInstances size"
2


As part of the CommandLineHandler allSubInstances there are eight LoadUpdatesCommandLineHandler still active. I would assume that a full GC should have collected them by now (some of them being created in May). In a headfull image they disappear quickly

I varied the execution a bit:

Pharo --headless RoamingHub.image eval --no-quit "[ Smalltalk garbageCollect. FileStream stdout print: CommandLineHandler allSubInstances size; lf. Smalltalk snapshot: false andQuit: true] fork"
0


Hypothesis:


* Command line handlers call >>#snapshot:andQuit:.
* Image resumes in this process
* New session created
* Command line handlers execute
* Calls snapshot:andQuit:
* Image resumes..
* New session created
* Command line handlers execute
...


Can this be true? I think the proposal to start the image differently would help here? Any comments/ideas?

holger

Reply | Threaded
Open this post in threaded view
|

Re: "Leaking" CommandLineHandler when running headless image

Guillermo Polito
Yes, that's the issue. I'm still convinced about that :)

On Tue, Sep 19, 2017 at 3:36 PM, Tim Mackinnon <[hidden email]> wrote:
There was a long post from Guille about how to sort this. It had to do with altering the priority of shutdown code - I'm not sure if we resolved it, but we should go back and see as it was well reasoned.

Tim 

Sent from my iPhone

On 19 Sep 2017, at 14:20, Denis Kudriashov <[hidden email]> wrote:

Hi.

It is probably related to issue 20309

2017-09-19 15:02 GMT+02:00 Holger Freyther <[hidden email]>:
Hi,

I am currently trying to run one of my images as non-root and related to that look into keeping changes in a different directory or not writing them at all. While looking at a write failure I saw some paths being logged that I used during CI to load code and should have been GCed.


Observation:


Pharo --headless My.image eval "Smalltalk garbageCollect. CommandLineHandler allSubInstances size"
84

Pharo --headfull My.image eval "Smalltalk garbageCollect. CommandLineHandler allSubInstances size"
2


As part of the CommandLineHandler allSubInstances there are eight LoadUpdatesCommandLineHandler still active. I would assume that a full GC should have collected them by now (some of them being created in May). In a headfull image they disappear quickly

I varied the execution a bit:

Pharo --headless RoamingHub.image eval --no-quit "[ Smalltalk garbageCollect. FileStream stdout print: CommandLineHandler allSubInstances size; lf. Smalltalk snapshot: false andQuit: true] fork"
0


Hypothesis:


* Command line handlers call >>#snapshot:andQuit:.
* Image resumes in this process
* New session created
* Command line handlers execute
* Calls snapshot:andQuit:
* Image resumes..
* New session created
* Command line handlers execute
...


Can this be true? I think the proposal to start the image differently would help here? Any comments/ideas?

holger




--

   

Guille Polito

Research Engineer

Centre de Recherche en Informatique, Signal et Automatique de Lille

CRIStAL - UMR 9189

French National Center for Scientific Research - http://www.cnrs.fr


Web: http://guillep.github.io

Phone: +33 06 52 70 66 13