Problem cleaning up image

Ok may be this is just an endless loop :)

Le 12/9/16 à 10:22, Marcus Denker a écrit :

Hello,

Yes, this is a bug that we really need to fix.

(sadly no information to add other thant that).

On 09 Sep 2016, at 15:16, Vitor Medina Cruz <[hidden email]> wrote:

Hello,

Isn't cleanup the headless image working right now? I know it has some problems, but I tried to:

curl get.pharo.org | bash 

./pharo Pharo.image clean --production

It runs for hours and don't complete. Last time I leave it for about 5 hours before I cancelled.

Regards,

Vitor

Hi,

I think it is not really realistic to expect this to work *after* loading code of others.

Why not 

-> do image clean

-> *then* load the packages you need?

Marcus

On 13 Sep 2016, at 03:19, Vitor Medina Cruz <[hidden email]> wrote:

Ok, I did some extra tests and I think I figured what is happening: the code I load on the image subject to clean has a startUp method that starts a Teapot server on port 8080. I tried various scenarios:

1- Load a fresh image from get.pharo.org and execute clean --> Ok

2- Load a fresh image from get.pharo.org only with the dependencies from my project (ConfigurationOfTwitterChallenge), one at a time, and execute clean --> Ok

3- Load a fresh image from get.pharo.org, apply ConfigurationOfTwitterChallenge and execute clean --> Fail

4- Load a fresh image from get.pharo.org, apply ConfigurationOfTwitterChallenge and execute eval "TwitterChallengeMain stopDefaultExecution. ImageCleaner cleanUpForProduction. Smalltalk snapshot: true andQuit: true." --> Ok (Assuming  ImageCleaner cleanUpForProduction. Smalltalk snapshot: true andQuit: true. does the same as clean --production)

It appears I cannot clean an Image with the server started. Question: is that an error or I really should not try to clean an image with a server on? If the former I open an issue on FogBuz and I will try on a not-headless vm to see if some error happen.

Regards,

Vitor

On Mon, Sep 12, 2016 at 6:59 AM, stepharo <[hidden email]> wrote:

Ok may be this is just an endless loop :)

Le 12/9/16 à 10:22, Marcus Denker a écrit :

Hello,

Yes, this is a bug that we really need to fix.

(sadly no information to add other thant that).

On 09 Sep 2016, at 15:16, Vitor Medina Cruz <[hidden email]> wrote:

Hello,

Isn't cleanup the headless image working right now? I know it has some problems, but I tried to:

curl get.pharo.org | bash 

./pharo Pharo.image clean --production

It runs for hours and don't complete. Last time I leave it for about 5 hours before I cancelled.

Regards,

Vitor

> On 13 Sep 2016, at 13:15, Vitor Medina Cruz <[hidden email]> wrote:
>
> Humm, why not? When I load the configuration it loads several other packages, wouldn't that need a cleanup too? When I first see that I could cleanup an image my first thought was to put it in a production state with my code, then I would cleanup to leave it as lean as possible. Isn't that correct?
>
> If I should not clean after load my packages why provide the functionality? Why not provide clean images to be used in production so that people could download and load it's ready to production code?
>

No, the #cleanUpForProduction just unloads tests, examples and things like that… it only touches loaded code in the sense that it
unloads all packages named -Tests and -Examples.

But if the external package happens to be structured in a way that this is bad, it would brake.

I never expected this to be run *after* more packages are loaded, just *before*.

What it does right now it this (you can see that it could be simpler if our packages are better structured):

cleanUpForProduction
        "self new cleanUpForProduction"
        "delete logo"
        PolymorphSystemSettings showDesktopLogo: false.

        self packagesForCleanUpInProduction do: [ :each | (MCPackage named: each) unload ].
        self helpPackages do: [ :each | (MCPackage named: each) unload ].
        self testPackages do: [ :each | (MCPackage named: each) unload ].
        self examplePackages do: [ :each | (MCPackage named: each) unload ].
        (MCPackage named: 'ScriptLoader40') unload.


        (MCPackage named: 'MonticelloMocks') unload.
        (MCPackage named: 'ToolsTest') unload.
        (MCPackage named: 'Announcements-Tests-Core') unload.
        (MCPackage named: 'AST-Tests-Core') unload.
        (MCPackage named: 'AST-Interpreter-Test') unload.
        (MCPackage named: 'Ring-Tests-Containers') unload.
        (MCPackage named: 'Ring-Tests-Kernel') unload.
        (MCPackage named: 'Ring-Tests-Monticello') unload.
        (MCPackage named: 'Regex-Tests-Core') unload.
        (MCPackage named: 'Refactoring-Tests-Changes') unload.
        (MCPackage named: 'Refactoring-Tests-Core') unload.
        (MCPackage named: 'Refactoring-Tests-Critics') unload.
        (MCPackage named: 'Refactoring-Tests-Environment') unload.
        (MCPackage named: 'FileSystem-Tests-Core') unload.
        (MCPackage named: 'FileSystem-Tests-Disk') unload.
        (MCPackage named: 'FileSystem-Tests-Memory') unload.

        (MCPackage named: 'GT-Tests-Inspector') unload.
        (MCPackage named: 'Glamour-Tests-Core') unload.
        (MCPackage named: 'Glamour-Tests-Morphic') unload.
        (MCPackage named: 'Glamour-Tests-Resources') unload.
        (MCPackage named: 'Glamour-Tests-Rubric') unload.

        (MCPackage named: 'Versionner-Core-Model') unload.
        (MCPackage named: 'Versionner-Core-DependenciesModel') unload.
        (MCPackage named: 'Versionner-Core-Announcements') unload.
        (MCPackage named: 'Versionner-Core-Commands') unload.
        (MCPackage named: 'Versionner-Spec-Browser') unload.
        (MCPackage named: 'Versionner-Tests-Core-Commands') unload.
        (MCPackage named: 'Versionner-Tests-Core-DependenciesModel') unload.
        (MCPackage named: 'Versionner-Tests-Core-Model') unload.
        (MCPackage named: 'Versionner-Tests-Resources') unload.

        self cleanUpForRelease.


        Marcus

> On 13 Sep 2016, at 18:54, Vitor Medina Cruz <[hidden email]> wrote:
>
> I understand. I thought it did more stuff, like remove necessary development tools or executing processes,

It does call #cleanUpForRelease, but this we call at *every* update on the CI, too. This calls  the #cleanUp method of all classes
and does some more (rehashing all dictionaries, restarting all? methods holding on to blocks…).

> I don't know, but considering its implementation it should affect only the image size, right?
>
Yes. Of course right now Cog is not compacting the somewhat empty allocated  memory regions, so e.g. the
47 MB image should be around 15 MB empty space and the cleanup will not shrink it.

But when this is fixed, you should see a change. A saved image is too a large extend methods and literals they
point to (especially for things like Tutorials). In the past the normal image was ca. 25MB, while the shrieked one
after cleanup for deployment maybe 15? I do not remember.

But that was already a bit of “moored law” time ago ;-)

        Marcus