Smalltalk › Pharo › Pharo Smalltalk Users

Image Cleaner no longer useful?

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

9 messages Options

Pharo Smalltalk Users mailing list

Image Cleaner no longer useful?

Hi, and sorry if this has already been discussed.

In Pharo 3, I used to be able to do:

ImageCleaner cleanUpForProduction

and that would correctly remove a lot of unneeded code in the image when preparing for a productive release.

Then I would save the cleaned image, and since it had been cleaned, it would be smaller than the original development image (say from 60 mb to 25 mb).

This process would take about 2-3 minutes.

Then, in Pharo 4 it seems some changes were made here, since the same process took a lot more, like 20 minutes.

When I first tried Pharo 5, I was happy to see that it was again down to 2-3 minutes.

And here comes my question...

Why is it that since some 2 months ago (maybe more), the Pharo 5 images once again require like 20 minutes to complete the cleanUpForProduction process?

Also, and this is the most relevant to me, it no longer reduces the image size.

So if I want to release a Pharo 5 version based on the latest images, I would be shipping a product that is more than twice the size of previous public versions of my software!

Thanks for any insights into this. If it is a known bug, I would like to know if there is any workaround, or alternative objects that I should be using for the process.

Also, if it would help for me to run any tests in particular to share and help in fixing this, please let me know.

Marcus Denker-4

Re: Image Cleaner no longer useful?

Hello,

Sorry for the late answer.

I think there are multiple problems…

1) slowness. This is strange that unloading packages is so slow. This looks like a bug to me.

2) image size in Pharo5: We have a bug right now that images *never* shrink. This means that if you save

an image once with 80MB, even if the GC cleans up, the saved image stays 80MB.

We should fix both….

On 25 Jan 2016, at 13:32, Maximiliano Tabacman via Pharo-users <[hidden email]> wrote:

From: Maximiliano Tabacman <[hidden email]>
Subject: Image Cleaner no longer useful?
Date: 25 January 2016 at 13:31:28 GMT+1
To: "[hidden email]" <[hidden email]>
Reply-To: Maximiliano Tabacman <[hidden email]>

Hi, and sorry if this has already been discussed.

In Pharo 3, I used to be able to do:

ImageCleaner cleanUpForProduction

and that would correctly remove a lot of unneeded code in the image when preparing for a productive release.
Then I would save the cleaned image, and since it had been cleaned, it would be smaller than the original development image (say from 60 mb to 25 mb).
This process would take about 2-3 minutes.

Then, in Pharo 4 it seems some changes were made here, since the same process took a lot more, like 20 minutes.

When I first tried Pharo 5, I was happy to see that it was again down to 2-3 minutes.

And here comes my question...
Why is it that since some 2 months ago (maybe more), the Pharo 5 images once again require like 20 minutes to complete the cleanUpForProduction process?

Also, and this is the most relevant to me, it no longer reduces the image size.
So if I want to release a Pharo 5 version based on the latest images, I would be shipping a product that is more than twice the size of previous public versions of my software!

Thanks for any insights into this. If it is a known bug, I would like to know if there is any workaround, or alternative objects that I should be using for the process.
Also, if it would help for me to run any tests in particular to share and help in fixing this, please let me know.

Pharo Smalltalk Users mailing list

Re: Image Cleaner no longer useful?

Thanks for the reply Marcus.

I just checked, and can confirm that the old process took only 2 minutes and converted a 58,2 MB image (includes Seaside and my web product) into a 26,4 MB production image.

By looking at PharoDebug.log, I can tell that this version is:

VM: Win32 - IX86 - 6.1 - NBCoInterpreter NativeBoost-CogPlugin-HolgerHansPeterFreyther.21 uuid: e0df6e2d-5694-40e2-8035-dc217200b424 Sep 25 2014

NBCogit NativeBoost-CogPlugin-HolgerHansPeterFreyther.21 uuid: e0df6e2d-5694-40e2-8035-dc217200b424 Sep 25 2014

https://github.com/pharo-project/pharo-vm.git Commit: 81b5d19917dcb78f22482a780deec48c53738396 Date: 2014-09-20 14:36:18 +0200 By: Esteban Lorenzano <[hidden email]> Jenkins build #14858

Image: Pharo5.0 [Latest update: #50491]

Hope that helps. This is Pharo 5, so the Image Cleaner must have broken at some point after this version.

Let me know if there is any additional information I can provide to help figure it out.

On Thursday, February 4, 2016 12:22 PM, Marcus Denker <[hidden email]> wrote:

Hello,

Sorry for the late answer.

I think there are multiple problems…

1) slowness. This is strange that unloading packages is so slow. This looks like a bug to me.

2) image size in Pharo5: We have a bug right now that images *never* shrink. This means that if you save

an image once with 80MB, even if the GC cleans up, the saved image stays 80MB.

We should fix both….

On 25 Jan 2016, at 13:32, Maximiliano Tabacman via Pharo-users <[hidden email]> wrote:

From: Maximiliano Tabacman <[hidden email]>
Subject: Image Cleaner no longer useful?
Date: 25 January 2016 at 13:31:28 GMT+1
To: "[hidden email]" <[hidden email]>
Reply-To: Maximiliano Tabacman <[hidden email]>

Hi, and sorry if this has already been discussed.

In Pharo 3, I used to be able to do:

ImageCleaner cleanUpForProduction

and that would correctly remove a lot of unneeded code in the image when preparing for a productive release.
Then I would save the cleaned image, and since it had been cleaned, it would be smaller than the original development image (say from 60 mb to 25 mb).
This process would take about 2-3 minutes.

Then, in Pharo 4 it seems some changes were made here, since the same process took a lot more, like 20 minutes.

When I first tried Pharo 5, I was happy to see that it was again down to 2-3 minutes.

And here comes my question...
Why is it that since some 2 months ago (maybe more), the Pharo 5 images once again require like 20 minutes to complete the cleanUpForProduction process?

Also, and this is the most relevant to me, it no longer reduces the image size.
So if I want to release a Pharo 5 version based on the latest images, I would be shipping a product that is more than twice the size of previous public versions of my software!

Thanks for any insights into this. If it is a known bug, I would like to know if there is any workaround, or alternative objects that I should be using for the process.
Also, if it would help for me to run any tests in particular to share and help in fixing this, please let me know.

Pharo Smalltalk Users mailing list

Re: Image Cleaner no longer useful?

I've got some additional details.

Using the old VM at: http://files.pharo.org/vm/pharo/win/Pharo-VM-win-stable.zip

I downloaded a fresh 50491 image at:

http://files.pharo.org/image/50-preSpur/50491.zip

Opened a playground and executed

ImageCleaner cleanUpForProduction

This was really fast, and in 2 minutes the process was complete.

Then I downloaded the 50492 image at:

http://files.pharo.org/image/50-preSpur/50492.zip

Did the same ImageCleaner cleanUpForProduction,

only this time is took about half an hour to do the same.

Hope that helps to catch the reason.

Thanks

On Friday, February 5, 2016 9:33 AM, Maximiliano Tabacman via Pharo-users <[hidden email]> wrote:

Thanks for the reply Marcus.

I just checked, and can confirm that the old process took only 2 minutes and converted a 58,2 MB image (includes Seaside and my web product) into a 26,4 MB production image.

By looking at PharoDebug.log, I can tell that this version is:

VM: Win32 - IX86 - 6.1 - NBCoInterpreter NativeBoost-CogPlugin-HolgerHansPeterFreyther.21 uuid: e0df6e2d-5694-40e2-8035-dc217200b424 Sep 25 2014

NBCogit NativeBoost-CogPlugin-HolgerHansPeterFreyther.21 uuid: e0df6e2d-5694-40e2-8035-dc217200b424 Sep 25 2014

https://github.com/pharo-project/pharo-vm.git Commit: 81b5d19917dcb78f22482a780deec48c53738396 Date: 2014-09-20 14:36:18 +0200 By: Esteban Lorenzano <[hidden email]> Jenkins build #14858

Image: Pharo5.0 [Latest update: #50491]

Hope that helps. This is Pharo 5, so the Image Cleaner must have broken at some point after this version.

Let me know if there is any additional information I can provide to help figure it out.

On Thursday, February 4, 2016 12:22 PM, Marcus Denker <[hidden email]> wrote:

Hello,

Sorry for the late answer.

I think there are multiple problems…

1) slowness. This is strange that unloading packages is so slow. This looks like a bug to me.

2) image size in Pharo5: We have a bug right now that images *never* shrink. This means that if you save

an image once with 80MB, even if the GC cleans up, the saved image stays 80MB.

We should fix both….

On 25 Jan 2016, at 13:32, Maximiliano Tabacman via Pharo-users <[hidden email]> wrote:

From: Maximiliano Tabacman <[hidden email]>
Subject: Image Cleaner no longer useful?
Date: 25 January 2016 at 13:31:28 GMT+1
To: "[hidden email]" <[hidden email]>
Reply-To: Maximiliano Tabacman <[hidden email]>

Hi, and sorry if this has already been discussed.

In Pharo 3, I used to be able to do:

ImageCleaner cleanUpForProduction

and that would correctly remove a lot of unneeded code in the image when preparing for a productive release.
Then I would save the cleaned image, and since it had been cleaned, it would be smaller than the original development image (say from 60 mb to 25 mb).
This process would take about 2-3 minutes.

Then, in Pharo 4 it seems some changes were made here, since the same process took a lot more, like 20 minutes.

When I first tried Pharo 5, I was happy to see that it was again down to 2-3 minutes.

And here comes my question...
Why is it that since some 2 months ago (maybe more), the Pharo 5 images once again require like 20 minutes to complete the cleanUpForProduction process?

Also, and this is the most relevant to me, it no longer reduces the image size.
So if I want to release a Pharo 5 version based on the latest images, I would be shipping a product that is more than twice the size of previous public versions of my software!

Thanks for any insights into this. If it is a known bug, I would like to know if there is any workaround, or alternative objects that I should be using for the process.
Also, if it would help for me to run any tests in particular to share and help in fixing this, please let me know.

kuszi

Pharo.ini access

Hello Everyone!

I wonder if there is a chance to access pharo.ini contents from inside the image

I'm lazy and I'd like to add my own settings (new keys) to pharo.ini. It would be nice to have these once the image is started so I don't have to create my own .ini and parse it.

So, is Pharo.ini key values are in the image in a class or somewhere?

I'd like to add:

gcDowloadDir = dirname

gcODBCsource = DSN
....

and so on :)

thanks

Robert

hernanmd

Re: Pharo.ini access

Hi Robert,

You have a nice INI parser in http://www.smalltalkhub.com/#!/~TorstenBergmann/INIFile

Cheers,

Hernán

2016-02-08 11:30 GMT-03:00 Robert Kuszinger <[hidden email]>:

Hello Everyone!

I wonder if there is a chance to access pharo.ini contents from inside the image

I'm lazy and I'd like to add my own settings (new keys) to pharo.ini. It would be nice to have these once the image is started so I don't have to create my own .ini and parse it.

So, is Pharo.ini key values are in the image in a class or somewhere?

I'd like to add:

gcDowloadDir = dirname
gcODBCsource = DSN
....

and so on :)

thanks
Robert

kuszi

Re: Pharo.ini access

Actually, I wanted to be lazier... But all right, I create my onw ini :)

Thanks!

Hernán Morales Durand <[hidden email]> ezt írta (időpont: 2016. febr. 8., H, 15:57):

Hi Robert,

You have a nice INI parser in http://www.smalltalkhub.com/#!/~TorstenBergmann/INIFile
Cheers,

Hernán

2016-02-08 11:30 GMT-03:00 Robert Kuszinger <[hidden email]>:
Hello Everyone!

I wonder if there is a chance to access pharo.ini contents from inside the image

I'm lazy and I'd like to add my own settings (new keys) to pharo.ini. It would be nice to have these once the image is started so I don't have to create my own .ini and parse it.

So, is Pharo.ini key values are in the image in a class or somewhere?

I'd like to add:

gcDowloadDir = dirname
gcODBCsource = DSN
....

and so on :)

thanks
Robert

Pharo Smalltalk Users mailing list

Re: Image Cleaner no longer useful?

In reply to this post by Pharo Smalltalk Users mailing list

With the latest Image (50651) and VM, the issue remains the same.

The most critical part for me is the fact that cleaning the image (at least using ImageCleaner) does not reduce its size.

I have a commercial product with users already using it, who obtain the product by downloading. This issue causes my product to be at least double the size in total because of the image not being reduced, and sometimes more.

I have no idea how to help solve this issue, but if it remains once Pharo 5 is released next month, that means I won't be able to use it for my product, and I would be stuck with older versions where the cleaning process worked better.

Should I be addressing this issue in another list? Maybe vote up how critical it is in some pending bugs list somewhere?

Don't get me wrong, I love Pharo and all the community work being done. I love the fact that I can get a humble commercial product out to the users.

I just think maybe other products are more server based and not affected by this issue, but for anyone creating products with Pharo that are to be downloaded by the customers it should be top priority.

Thanks for Pharo, and thanks for any help or guidance you may be able to provide on this matter.

On Sunday, February 7, 2016 1:10 PM, Maximiliano Tabacman via Pharo-users <[hidden email]> wrote:

I've got some additional details.

Using the old VM at: http://files.pharo.org/vm/pharo/win/Pharo-VM-win-stable.zip

I downloaded a fresh 50491 image at:

http://files.pharo.org/image/50-preSpur/50491.zip

Opened a playground and executed

ImageCleaner cleanUpForProduction

This was really fast, and in 2 minutes the process was complete.

Then I downloaded the 50492 image at:

http://files.pharo.org/image/50-preSpur/50492.zip

Did the same ImageCleaner cleanUpForProduction,

only this time is took about half an hour to do the same.

Hope that helps to catch the reason.

Thanks

On Friday, February 5, 2016 9:33 AM, Maximiliano Tabacman via Pharo-users <[hidden email]> wrote:

Thanks for the reply Marcus.

I just checked, and can confirm that the old process took only 2 minutes and converted a 58,2 MB image (includes Seaside and my web product) into a 26,4 MB production image.

By looking at PharoDebug.log, I can tell that this version is:

VM: Win32 - IX86 - 6.1 - NBCoInterpreter NativeBoost-CogPlugin-HolgerHansPeterFreyther.21 uuid: e0df6e2d-5694-40e2-8035-dc217200b424 Sep 25 2014

NBCogit NativeBoost-CogPlugin-HolgerHansPeterFreyther.21 uuid: e0df6e2d-5694-40e2-8035-dc217200b424 Sep 25 2014

https://github.com/pharo-project/pharo-vm.git Commit: 81b5d19917dcb78f22482a780deec48c53738396 Date: 2014-09-20 14:36:18 +0200 By: Esteban Lorenzano <[hidden email]> Jenkins build #14858

Image: Pharo5.0 [Latest update: #50491]

Hope that helps. This is Pharo 5, so the Image Cleaner must have broken at some point after this version.

Let me know if there is any additional information I can provide to help figure it out.

On Thursday, February 4, 2016 12:22 PM, Marcus Denker <[hidden email]> wrote:

Hello,

Sorry for the late answer.

I think there are multiple problems…

1) slowness. This is strange that unloading packages is so slow. This looks like a bug to me.

2) image size in Pharo5: We have a bug right now that images *never* shrink. This means that if you save

an image once with 80MB, even if the GC cleans up, the saved image stays 80MB.

We should fix both….

On 25 Jan 2016, at 13:32, Maximiliano Tabacman via Pharo-users <[hidden email]> wrote:

From: Maximiliano Tabacman <[hidden email]>
Subject: Image Cleaner no longer useful?
Date: 25 January 2016 at 13:31:28 GMT+1
To: "[hidden email]" <[hidden email]>
Reply-To: Maximiliano Tabacman <[hidden email]>

Hi, and sorry if this has already been discussed.

In Pharo 3, I used to be able to do:

ImageCleaner cleanUpForProduction

and that would correctly remove a lot of unneeded code in the image when preparing for a productive release.
Then I would save the cleaned image, and since it had been cleaned, it would be smaller than the original development image (say from 60 mb to 25 mb).
This process would take about 2-3 minutes.

Then, in Pharo 4 it seems some changes were made here, since the same process took a lot more, like 20 minutes.

When I first tried Pharo 5, I was happy to see that it was again down to 2-3 minutes.

And here comes my question...
Why is it that since some 2 months ago (maybe more), the Pharo 5 images once again require like 20 minutes to complete the cleanUpForProduction process?

Also, and this is the most relevant to me, it no longer reduces the image size.
So if I want to release a Pharo 5 version based on the latest images, I would be shipping a product that is more than twice the size of previous public versions of my software!

Thanks for any insights into this. If it is a known bug, I would like to know if there is any workaround, or alternative objects that I should be using for the process.
Also, if it would help for me to run any tests in particular to share and help in fixing this, please let me know.

Stephan Eggermont-3

Re: Image Cleaner no longer useful?

Hello Maximiliano

On 20-03-16 13:29, Maximiliano Tabacman via Pharo-users wrote:
> With the latest Image (50651) and VM, the issue remains the same.

As Marcus wrote, that is a well-known problem.

At the moment, there is one higher priority issue
on the image side, and it might well be related.
Furthermore, it might be related to the vm switch
to spur, and stabilizing that is higher priority and
currently takes a lot of effort, as you can see on
pharoproject.slack.com, pharo-dev and vm-dev

Stephan