Smalltalk › Squeak › Squeak - Dev

About unloading of packages in the most recent Squeak 4.1 trunk

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

51 messages Options

123

Hannes Hirzel

About unloading of packages in the most recent Squeak 4.1 trunk

Hello

Could somebody please post the code snippet which unloads all the
packages in Squeak 4.1 trunk latest update which unload and reload
cleanly?

--Hannes

Edgar De Cleene

Re: About unloading of packages in the most recent Squeak 4.1 trunk

Here the thing i made for 3.10, Andreas improve and i use for
SeasideLight3.0 , with this you rip 3 mb from ready to run image (Seaside
3.0)

Edgar

SmalltalkImage-unloadAllKnownPackages.st (3K) Download Attachment

Casey Ransberger-2

Re: About unloading of packages in the most recent Squeak 4.1 trunk

In reply to this post by Hannes Hirzel

It was (may have changed with the SmalltalkImage refactor)

Smalltalk unloadAllKnownPackages.

On Aug 21, 2010, at 4:50 AM, Hannes Hirzel <[hidden email]> wrote:

> Hello
>
> Could somebody please post the code snippet which unloads all the
> packages in Squeak 4.1 trunk latest update which unload and reload
> cleanly?
>
> --Hannes
>

Hannes Hirzel

Re: About unloading of packages in the most recent Squeak 4.1 trunk

Thank you Casey, this was the 'hook' I was looking for.

In fact it has changed to SmalltalkImage as you write.

So I downloaded a trunk image (20thAugust)
http://ftp.squeak.org/trunk/Squeak4.2-10382-alpha.zip

and tried
SmalltalkImage unloadAllKnownPackages

and get

SmalltalkImage doesNotUnderstand: #unloadAllKnownPackages

though if I search for the method #unloadAllKnownPackages it is there.

Any ideas how what to do?

My aim
======

I want to do a retest of what currently works in terms of unloading. I
remember Andreas announcing this method #unloadAllKnownPackages when
publishing 4.1.
May aim is to do a retest and see if this still works.

--Hannes

On 8/21/10, Casey Ransberger <[hidden email]> wrote:

> It was (may have changed with the SmalltalkImage refactor)
>
> Smalltalk unloadAllKnownPackages.
>
> On Aug 21, 2010, at 4:50 AM, Hannes Hirzel <[hidden email]> wrote:
>
>> Hello
>>
>> Could somebody please post the code snippet which unloads all the
>> packages in Squeak 4.1 trunk latest update which unload and reload
>> cleanly?
>>
>> --Hannes
>>
>
>

Tobias Pape

Re: About unloading of packages in the most recent Squeak 4.1 trunk

Hi,

Am 2010-08-23 um 11:06 schrieb Hannes Hirzel:

> and tried
> SmalltalkImage unloadAllKnownPackages

Do
Smalltalk unloadAllKnownPackages

instead, since Smalltalk is the sole instance
of SmalltalkImage.

So Long,
-Tobias

Tobias Pape

Re: About unloading of packages in the most recent Squeak 4.1 trunk

Am 2010-08-23 um 11:14 schrieb Tobias Pape:

> Hi,
>
> Am 2010-08-23 um 11:06 schrieb Hannes Hirzel:
>
>> and tried
>> SmalltalkImage unloadAllKnownPackages
>
> Do
> Smalltalk unloadAllKnownPackages
>
> instead, since Smalltalk is the sole instance
> of SmalltalkImage.
>
> So Long,
> -Tobias
>

Addendum:
It works.
resulting file size: 10411140, about 10,4 MB.
Nice.

Btw, this script probably needs updating, The Help
browser, eg, does not get unloaded (same for TrueType)

So Long,
-Tobias

Hannes Hirzel

Re: About unloading of packages in the most recent Squeak 4.1 trunk

On 8/23/10, Tobias Pape <[hidden email]> wrote:

>
> Am 2010-08-23 um 11:14 schrieb Tobias Pape:
>
>> Hi,
>>
>> Am 2010-08-23 um 11:06 schrieb Hannes Hirzel:
>>
>>> and tried
>>> SmalltalkImage unloadAllKnownPackages
>>
>> Do
>> Smalltalk unloadAllKnownPackages
>>
>> instead, since Smalltalk is the sole instance
>> of SmalltalkImage.

Thank you for point this out.

In fact Smalltalk is the global variable which references
SmalltalkImage current

So I can do as well

SmalltalkImage current unloadAllKnownPackages

Which I did

Squeak4.2-10382-alpha.image went from 16998KB

down to
10277kB
(file sizes measured on Windows machine)

>> So Long,
>> -Tobias
>>

> Addendum:
> It works.
> resulting file size: 10411140, about 10,4 MB.
> Nice.
>
> Btw, this script probably needs updating, The Help
> browser, eg, does not get unloaded (same for TrueType)

Yes, I tried it for the HelpSystem

I added three more entries to the literal array in the method

"Go unloading"
#( 'ReleaseBuilder' 'ScriptLoader'
'311Deprecated' '39Deprecated'
'Universes' 'SMLoader' 'SMBase' 'Installer-Core'
'VersionNumberTests' 'VersionNumber'
'Services-Base' 'PreferenceBrowser' 'Nebraska'
'ToolBuilder-MVC' 'ST80'
'CollectionsTests' 'GraphicsTests' 'KernelTests' 'MorphicTests'
'MultilingualTests' 'NetworkTests' 'ToolsTests' 'TraitsTests'
'SystemChangeNotification-Tests' 'FlexibleVocabularies'
'EToys' 'Protocols' 'XML-Parser' 'Tests' 'SUnitGUI',
'HelpSystem-Core', 'HelpSystem-Tests', 'HelpSystem-Project'
) do:[:pkgName| (MCPackage named: pkgName) unload].

The image is now
10174kB (=100kB less10277kB).

Probably not worth the effort at this time.

Questions
1) Which are the packages which are big and easy to unload.....?
2) Who does an attempt to unload the TrueType?

--Hannes

Levente Uzonyi-2

Re: About unloading of packages in the most recent Squeak 4.1 trunk

On Mon, 23 Aug 2010, Hannes Hirzel wrote:

> On 8/23/10, Tobias Pape <[hidden email]> wrote:
>>
>> Am 2010-08-23 um 11:14 schrieb Tobias Pape:
>>
>>> Hi,
>>>
>>> Am 2010-08-23 um 11:06 schrieb Hannes Hirzel:
>>>
>>>> and tried
>>>> SmalltalkImage unloadAllKnownPackages
>>>
>>> Do
>>> Smalltalk unloadAllKnownPackages
>>>
>>> instead, since Smalltalk is the sole instance
>>> of SmalltalkImage.
>
> Thank you for point this out.
>
> In fact Smalltalk is the global variable which references
> SmalltalkImage current
>
> So I can do as well
>
> SmalltalkImage current unloadAllKnownPackages
>
>
> Which I did
>
> Squeak4.2-10382-alpha.image went from 16998KB

Hm, I think Casey forgot to do [Smalltalk cleanUp: false] before saving
the image. Doing that saves 2.8MB.

Levente

>
> down to
> 10277kB
> (file sizes measured on Windows machine)
>
>>> So Long,
>>> -Tobias
>>>
>
>
>
>> Addendum:
>> It works.
>> resulting file size: 10411140, about 10,4 MB.
>> Nice.
>>
>> Btw, this script probably needs updating, The Help
>> browser, eg, does not get unloaded (same for TrueType)
>
> Yes, I tried it for the HelpSystem
>
> I added three more entries to the literal array in the method
>
> "Go unloading"
> #( 'ReleaseBuilder' 'ScriptLoader'
> '311Deprecated' '39Deprecated'
> 'Universes' 'SMLoader' 'SMBase' 'Installer-Core'
> 'VersionNumberTests' 'VersionNumber'
> 'Services-Base' 'PreferenceBrowser' 'Nebraska'
> 'ToolBuilder-MVC' 'ST80'
> 'CollectionsTests' 'GraphicsTests' 'KernelTests' 'MorphicTests'
> 'MultilingualTests' 'NetworkTests' 'ToolsTests' 'TraitsTests'
> 'SystemChangeNotification-Tests' 'FlexibleVocabularies'
> 'EToys' 'Protocols' 'XML-Parser' 'Tests' 'SUnitGUI',
> 'HelpSystem-Core', 'HelpSystem-Tests', 'HelpSystem-Project'
> ) do:[:pkgName| (MCPackage named: pkgName) unload].
>
>
> The image is now
> 10174kB (=100kB less10277kB).
>
> Probably not worth the effort at this time.
>
> Questions
> 1) Which are the packages which are big and easy to unload.....?
> 2) Who does an attempt to unload the TrueType?
>
>
> --Hannes
>
>

Casey Ransberger-2

Re: About unloading of packages in the most recent Squeak 4.1 trunk

Levente: totally. I mainly just wanted

a) image can start on Cog w/o segfaulting

b) doesn't need a long lengthy set of updates before being caught up with trunk

The cleanup methods will be run before we ship anyway:)

On Aug 23, 2010, at 4:29 AM, Levente Uzonyi <[hidden email]> wrote:

> On Mon, 23 Aug 2010, Hannes Hirzel wrote:
>
>> On 8/23/10, Tobias Pape <[hidden email]> wrote:
>>>
>>> Am 2010-08-23 um 11:14 schrieb Tobias Pape:
>>>
>>>> Hi,
>>>>
>>>> Am 2010-08-23 um 11:06 schrieb Hannes Hirzel:
>>>>
>>>>> and tried
>>>>> SmalltalkImage unloadAllKnownPackages
>>>>
>>>> Do
>>>> Smalltalk unloadAllKnownPackages
>>>>
>>>> instead, since Smalltalk is the sole instance
>>>> of SmalltalkImage.
>>
>> Thank you for point this out.
>>
>> In fact Smalltalk is the global variable which references
>> SmalltalkImage current
>>
>> So I can do as well
>>
>> SmalltalkImage current unloadAllKnownPackages
>>
>>
>> Which I did
>>
>> Squeak4.2-10382-alpha.image went from 16998KB
>
> Hm, I think Casey forgot to do [Smalltalk cleanUp: false] before saving the image. Doing that saves 2.8MB.
>
>
> Levente
>
>>
>> down to
>> 10277kB
>> (file sizes measured on Windows machine)
>>
>>>> So Long,
>>>> -Tobias
>>>>
>>
>>
>>
>>> Addendum:
>>> It works.
>>> resulting file size: 10411140, about 10,4 MB.
>>> Nice.
>>>
>>> Btw, this script probably needs updating, The Help
>>> browser, eg, does not get unloaded (same for TrueType)
>>
>> Yes, I tried it for the HelpSystem
>>
>> I added three more entries to the literal array in the method
>>
>> "Go unloading"
>> #( 'ReleaseBuilder' 'ScriptLoader'
>> '311Deprecated' '39Deprecated'
>> 'Universes' 'SMLoader' 'SMBase' 'Installer-Core'
>> 'VersionNumberTests' 'VersionNumber'
>> 'Services-Base' 'PreferenceBrowser' 'Nebraska'
>> 'ToolBuilder-MVC' 'ST80'
>> 'CollectionsTests' 'GraphicsTests' 'KernelTests' 'MorphicTests'
>> 'MultilingualTests' 'NetworkTests' 'ToolsTests' 'TraitsTests'
>> 'SystemChangeNotification-Tests' 'FlexibleVocabularies'
>> 'EToys' 'Protocols' 'XML-Parser' 'Tests' 'SUnitGUI',
>> 'HelpSystem-Core', 'HelpSystem-Tests', 'HelpSystem-Project'
>> ) do:[:pkgName| (MCPackage named: pkgName) unload].
>>
>>
>> The image is now
>> 10174kB (=100kB less10277kB).
>>
>> Probably not worth the effort at this time.
>>
>> Questions
>> 1) Which are the packages which are big and easy to unload.....?
>> 2) Who does an attempt to unload the TrueType?
>>
>>
>> --Hannes
>>
>>
>

Tobias Pape

Re: About unloading of packages in the most recent Squeak 4.1 trunk

In reply to this post by Tobias Pape

>>
>> Smalltalk unloadAllKnownPackages
[…]

> Addendum:
> It works.
> resulting file size: 10411140, about 10,4 MB.
> Nice.
>
> Btw, this script probably needs updating, The Help
> browser, eg, does not get unloaded (same for TrueType)
>
> So Long,
> -Tobias
>

Addendum 2:

After unload, updating Trunk no longer works,
first error: trait no longer available.

just a note ;)

so long,
-Tobias

Edgar De Cleene

Re: About unloading of packages in the most recent Squeak 4.1 trunk

On 8/23/10 2:49 PM, "Tobias Pape" <[hidden email]> wrote:

> Addendum 2:
>
> After unload, updating Trunk no longer works,
> first error: trait no longer available.
>
> just a note ;)
>
> so long,
> -Tobias

It's why I modify the method.
Give a try to
http://ftp.squeak.org/Experiments/SqueakCore4.2-10382-alpha.zip

Edgar

Casey Ransberger-2

Re: About unloading of packages in the most recent Squeak 4.1 trunk

In reply to this post by Tobias Pape

Yeah, which is why we currently have to work in the monolithic image. Once we've agreed on how we're going to load "community supported packages," we'll be able to move them out of the core image and out of the main trunk repository, which should solve this problem (I think, but I may not understand the issue: if I do, the problem is that we're trying to update things that aren't in the image after the unload.)

On Aug 23, 2010, at 10:49 AM, Tobias Pape <[hidden email]> wrote:

> After unload, updating Trunk no longer works,
> first error: trait no longer available.
>
> just a note ;)
>
> so long,
> -Tobias

Tobias Pape

Re: About unloading of packages in the most recent Squeak 4.1 trunk

In reply to this post by Casey Ransberger-2

Hi,
Am 2010-08-23 um 19:40 schrieb Casey Ransberger:

>> Hm, I think Casey forgot to do [Smalltalk cleanUp: false] before saving the image. Doing that saves 2.8MB.
>

makes no difference after unloading.
wich is somewhat surprising to me.

so long,
-Tobias

Levente Uzonyi-2

Re: About unloading of packages in the most recent Squeak 4.1 trunk

In reply to this post by Edgar De Cleene

On Mon, 23 Aug 2010, Edgar J. De Cleene wrote:

>
>
>
> On 8/23/10 2:49 PM, "Tobias Pape" <[hidden email]> wrote:
>
>> Addendum 2:
>>
>> After unload, updating Trunk no longer works,
>> first error: trait no longer available.
>>
>> just a note ;)
>>
>> so long,
>> -Tobias
>
> It's why I modify the method.
> Give a try to
> http://ftp.squeak.org/Experiments/SqueakCore4.2-10382-alpha.zip

I did, but it's not better at all. You simply didn't unload Traits, so
when the updater is trying to load 311-Deprecated it doesn't miss them.

The problem with updating after unloading is that the updater mechanism
has no information of package dependencies, so it will try to load
packages randomly which obviously won't work.

Levente

>
> Edgar
>
>
>
>

Levente Uzonyi-2

Re: About unloading of packages in the most recent Squeak 4.1 trunk

In reply to this post by Casey Ransberger-2

On Mon, 23 Aug 2010, Casey Ransberger wrote:

> Levente: totally. I mainly just wanted
>
> a) image can start on Cog w/o segfaulting
>
> b) doesn't need a long lengthy set of updates before being caught up with trunk
>
> The cleanup methods will be run before we ship anyway:)

That's ok, but these prebuilt images could be a bit smaller this way.

Levente

>
> On Aug 23, 2010, at 4:29 AM, Levente Uzonyi <[hidden email]> wrote:
>
>> On Mon, 23 Aug 2010, Hannes Hirzel wrote:
>>
>>> On 8/23/10, Tobias Pape <[hidden email]> wrote:
>>>>
>>>> Am 2010-08-23 um 11:14 schrieb Tobias Pape:
>>>>
>>>>> Hi,
>>>>>
>>>>> Am 2010-08-23 um 11:06 schrieb Hannes Hirzel:
>>>>>
>>>>>> and tried
>>>>>> SmalltalkImage unloadAllKnownPackages
>>>>>
>>>>> Do
>>>>> Smalltalk unloadAllKnownPackages
>>>>>
>>>>> instead, since Smalltalk is the sole instance
>>>>> of SmalltalkImage.
>>>
>>> Thank you for point this out.
>>>
>>> In fact Smalltalk is the global variable which references
>>> SmalltalkImage current
>>>
>>> So I can do as well
>>>
>>> SmalltalkImage current unloadAllKnownPackages
>>>
>>>
>>> Which I did
>>>
>>> Squeak4.2-10382-alpha.image went from 16998KB
>>
>> Hm, I think Casey forgot to do [Smalltalk cleanUp: false] before saving the image. Doing that saves 2.8MB.
>>
>>
>> Levente
>>
>>>
>>> down to
>>> 10277kB
>>> (file sizes measured on Windows machine)
>>>
>>>>> So Long,
>>>>> -Tobias
>>>>>
>>>
>>>
>>>
>>>> Addendum:
>>>> It works.
>>>> resulting file size: 10411140, about 10,4 MB.
>>>> Nice.
>>>>
>>>> Btw, this script probably needs updating, The Help
>>>> browser, eg, does not get unloaded (same for TrueType)
>>>
>>> Yes, I tried it for the HelpSystem
>>>
>>> I added three more entries to the literal array in the method
>>>
>>> "Go unloading"
>>> #( 'ReleaseBuilder' 'ScriptLoader'
>>> '311Deprecated' '39Deprecated'
>>> 'Universes' 'SMLoader' 'SMBase' 'Installer-Core'
>>> 'VersionNumberTests' 'VersionNumber'
>>> 'Services-Base' 'PreferenceBrowser' 'Nebraska'
>>> 'ToolBuilder-MVC' 'ST80'
>>> 'CollectionsTests' 'GraphicsTests' 'KernelTests' 'MorphicTests'
>>> 'MultilingualTests' 'NetworkTests' 'ToolsTests' 'TraitsTests'
>>> 'SystemChangeNotification-Tests' 'FlexibleVocabularies'
>>> 'EToys' 'Protocols' 'XML-Parser' 'Tests' 'SUnitGUI',
>>> 'HelpSystem-Core', 'HelpSystem-Tests', 'HelpSystem-Project'
>>> ) do:[:pkgName| (MCPackage named: pkgName) unload].
>>>
>>>
>>> The image is now
>>> 10174kB (=100kB less10277kB).
>>>
>>> Probably not worth the effort at this time.
>>>
>>> Questions
>>> 1) Which are the packages which are big and easy to unload.....?
>>> 2) Who does an attempt to unload the TrueType?
>>>
>>>
>>> --Hannes
>>>
>>>
>>
>
>

Levente Uzonyi-2

Re: About unloading of packages in the most recent Squeak 4.1 trunk

In reply to this post by Tobias Pape

On Mon, 23 Aug 2010, Tobias Pape wrote:

> Hi,
> Am 2010-08-23 um 19:40 schrieb Casey Ransberger:
>
>>> Hm, I think Casey forgot to do [Smalltalk cleanUp: false] before saving the image. Doing that saves 2.8MB.
>>
>
> makes no difference after unloading.
> wich is somewhat surprising to me.

But it's ok. [Smalltalk cleanUp: false] just releases cached objects.
Some of these objects are released when the packages are unloaded, others
are cleaned by SmalltalkImage >> #unloadAllKnownPackages, like:
Undeclared removeUnreferencedKeys.
StandardScriptingSystem initialize.
MCFileBasedRepository flushAllCaches.
MCDefinition clearInstances.
Behavior flushObsoleteSubclasses.
ChangeSet current clear.
etc.

Levente

>
> so long,
> -Tobias
>

Bert Freudenberg

Re: About unloading of packages in the most recent Squeak 4.1 trunk

In reply to this post by Levente Uzonyi-2

On 24.08.2010, at 00:03, Levente Uzonyi wrote:

> The problem with updating after unloading is that the updater mechanism has no information of package dependencies, so it will try to load packages randomly which obviously won't work.

It does not load packages randomly, but in the order defined in the update config map.

Since updating should not bring in the unloaded packages we would need to inform the updater to skip them.

- Bert -

Levente Uzonyi-2

Re: About unloading of packages in the most recent Squeak 4.1 trunk

On Tue, 24 Aug 2010, Bert Freudenberg wrote:

> On 24.08.2010, at 00:03, Levente Uzonyi wrote:
>
>> The problem with updating after unloading is that the updater mechanism has no information of package dependencies, so it will try to load packages randomly which obviously won't work.
>
> It does not load packages randomly, but in the order defined in the update config map.

That's right, but it's still random from the package dependency POV.

>
> Since updating should not bring in the unloaded packages we would need to inform the updater to skip them.

Right.

Levente

>
> - Bert -
>
>
>

Bert Freudenberg

Re: About unloading of packages in the most recent Squeak 4.1 trunk

On 24.08.2010, at 00:34, Levente Uzonyi wrote:

> On Tue, 24 Aug 2010, Bert Freudenberg wrote:
>
>> On 24.08.2010, at 00:03, Levente Uzonyi wrote:
>>
>>> The problem with updating after unloading is that the updater mechanism has no information of package dependencies, so it will try to load packages randomly which obviously won't work.
>>
>> It does not load packages randomly, but in the order defined in the update config map.
>
> That's right, but it's still random from the package dependency POV.

What makes you think so? The order is chosen (manually) so that dependent packages are loaded before depending packages.

- Bert -

>> Since updating should not bring in the unloaded packages we would need to inform the updater to skip them.
>
> Right.
>
>
> Levente
>
>>
>> - Bert -
>>
>>
>>
>

Levente Uzonyi-2

Re: About unloading of packages in the most recent Squeak 4.1 trunk

On Tue, 24 Aug 2010, Bert Freudenberg wrote:

>
> On 24.08.2010, at 00:34, Levente Uzonyi wrote:
>
>> On Tue, 24 Aug 2010, Bert Freudenberg wrote:
>>
>>> On 24.08.2010, at 00:03, Levente Uzonyi wrote:
>>>
>>>> The problem with updating after unloading is that the updater mechanism has no information of package dependencies, so it will try to load packages randomly which obviously won't work.
>>>
>>> It does not load packages randomly, but in the order defined in the update config map.
>>
>> That's right, but it's still random from the package dependency POV.
>
> What makes you think so? The order is chosen (manually) so that dependent packages are loaded before depending packages.

If that would be the case, then updating an "unloaded" image would reload
all unloaded packages. But that doesn't work. For example 311Deprecated
depends on Traits, but Traits is at the end of the list while
311Deprecated is the first non-dummy package.

Levente

>
> - Bert -
>
>>> Since updating should not bring in the unloaded packages we would need to inform the updater to skip them.
>>
>> Right.
>>
>>
>> Levente
>>
>>>
>>> - Bert -
>>>
>>>
>>>
>>
>
>
>
>

123