About unloading of packages in the most recent Squeak 4.1 trunk

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
51 messages Options
123
Reply | Threaded
Open this post in threaded view
|

About unloading of packages in the most recent Squeak 4.1 trunk

Hannes Hirzel
Hello

Could somebody please post  the code snippet which unloads all the
packages in Squeak 4.1 trunk latest update which unload and reload
cleanly?

--Hannes

Reply | Threaded
Open this post in threaded view
|

Re: About unloading of packages in the most recent Squeak 4.1 trunk

Edgar De Cleene

Here the thing i made for 3.10, Andreas improve and i use for
SeasideLight3.0 , with this you rip 3 mb from ready to run image (Seaside
3.0)

Edgar




SmalltalkImage-unloadAllKnownPackages.st (3K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: About unloading of packages in the most recent Squeak 4.1 trunk

Casey Ransberger-2
In reply to this post by Hannes Hirzel
It was (may have changed with the SmalltalkImage refactor)

Smalltalk unloadAllKnownPackages.

On Aug 21, 2010, at 4:50 AM, Hannes Hirzel <[hidden email]> wrote:

> Hello
>
> Could somebody please post  the code snippet which unloads all the
> packages in Squeak 4.1 trunk latest update which unload and reload
> cleanly?
>
> --Hannes
>

Reply | Threaded
Open this post in threaded view
|

Re: About unloading of packages in the most recent Squeak 4.1 trunk

Hannes Hirzel
Thank you Casey, this was the 'hook' I was looking for.

In fact it has changed to SmalltalkImage as you write.

So I downloaded a trunk image (20thAugust)
http://ftp.squeak.org/trunk/Squeak4.2-10382-alpha.zip

and tried
    SmalltalkImage unloadAllKnownPackages

and get

SmalltalkImage  doesNotUnderstand: #unloadAllKnownPackages

though if I search for the method #unloadAllKnownPackages it is there.

Any ideas how what to do?


My aim
======

I want to do a retest of what currently works in terms of unloading. I
remember Andreas announcing this method #unloadAllKnownPackages when
publishing 4.1.
May aim is to do a retest and see if this still works.

--Hannes


On 8/21/10, Casey Ransberger <[hidden email]> wrote:

> It was (may have changed with the SmalltalkImage refactor)
>
> Smalltalk unloadAllKnownPackages.
>
> On Aug 21, 2010, at 4:50 AM, Hannes Hirzel <[hidden email]> wrote:
>
>> Hello
>>
>> Could somebody please post  the code snippet which unloads all the
>> packages in Squeak 4.1 trunk latest update which unload and reload
>> cleanly?
>>
>> --Hannes
>>
>
>

Reply | Threaded
Open this post in threaded view
|

Re: About unloading of packages in the most recent Squeak 4.1 trunk

Tobias Pape
Hi,

Am 2010-08-23 um 11:06 schrieb Hannes Hirzel:

> and tried
>    SmalltalkImage unloadAllKnownPackages

Do
        Smalltalk unloadAllKnownPackages

instead, since Smalltalk is the sole instance
of SmalltalkImage.

So Long,
        -Tobias

Reply | Threaded
Open this post in threaded view
|

Re: About unloading of packages in the most recent Squeak 4.1 trunk

Tobias Pape

Am 2010-08-23 um 11:14 schrieb Tobias Pape:

> Hi,
>
> Am 2010-08-23 um 11:06 schrieb Hannes Hirzel:
>
>> and tried
>>   SmalltalkImage unloadAllKnownPackages
>
> Do
> Smalltalk unloadAllKnownPackages
>
> instead, since Smalltalk is the sole instance
> of SmalltalkImage.
>
> So Long,
> -Tobias
>

Addendum:
It works.
        resulting file size: 10411140, about 10,4 MB.
Nice.

Btw, this script probably needs updating, The Help
browser, eg, does not get unloaded (same for TrueType)

So Long,
        -Tobias

Reply | Threaded
Open this post in threaded view
|

Re: About unloading of packages in the most recent Squeak 4.1 trunk

Hannes Hirzel
On 8/23/10, Tobias Pape <[hidden email]> wrote:

>
> Am 2010-08-23 um 11:14 schrieb Tobias Pape:
>
>> Hi,
>>
>> Am 2010-08-23 um 11:06 schrieb Hannes Hirzel:
>>
>>> and tried
>>>   SmalltalkImage unloadAllKnownPackages
>>
>> Do
>> Smalltalk unloadAllKnownPackages
>>
>> instead, since Smalltalk is the sole instance
>> of SmalltalkImage.

Thank you for point this out.

In fact Smalltalk is the global variable which references
      SmalltalkImage current

So I can do as well

      SmalltalkImage current unloadAllKnownPackages


Which I did

Squeak4.2-10382-alpha.image went from 16998KB

down to
10277kB
(file sizes measured on Windows machine)

>> So Long,
>> -Tobias
>>



> Addendum:
> It works.
> resulting file size: 10411140, about 10,4 MB.
> Nice.
>
> Btw, this script probably needs updating, The Help
> browser, eg, does not get unloaded (same for TrueType)

Yes, I tried it for the HelpSystem

I added three more entries to the literal array in the method

"Go unloading"
        #( 'ReleaseBuilder' 'ScriptLoader'
                '311Deprecated' '39Deprecated'
                'Universes' 'SMLoader' 'SMBase' 'Installer-Core'
                'VersionNumberTests' 'VersionNumber'
                'Services-Base' 'PreferenceBrowser' 'Nebraska'
                'ToolBuilder-MVC' 'ST80'
                'CollectionsTests' 'GraphicsTests' 'KernelTests'  'MorphicTests'
                'MultilingualTests' 'NetworkTests' 'ToolsTests' 'TraitsTests'
                'SystemChangeNotification-Tests' 'FlexibleVocabularies'
                'EToys' 'Protocols' 'XML-Parser' 'Tests' 'SUnitGUI',
'HelpSystem-Core', 'HelpSystem-Tests', 'HelpSystem-Project'
        ) do:[:pkgName| (MCPackage named: pkgName) unload].


The image is now
10174kB (=100kB less10277kB).

Probably not worth the effort at this time.

Questions
1)  Which are the packages which are big and easy to unload.....?
2) Who does an attempt to unload the TrueType?


--Hannes

Reply | Threaded
Open this post in threaded view
|

Re: About unloading of packages in the most recent Squeak 4.1 trunk

Levente Uzonyi-2
On Mon, 23 Aug 2010, Hannes Hirzel wrote:

> On 8/23/10, Tobias Pape <[hidden email]> wrote:
>>
>> Am 2010-08-23 um 11:14 schrieb Tobias Pape:
>>
>>> Hi,
>>>
>>> Am 2010-08-23 um 11:06 schrieb Hannes Hirzel:
>>>
>>>> and tried
>>>>   SmalltalkImage unloadAllKnownPackages
>>>
>>> Do
>>> Smalltalk unloadAllKnownPackages
>>>
>>> instead, since Smalltalk is the sole instance
>>> of SmalltalkImage.
>
> Thank you for point this out.
>
> In fact Smalltalk is the global variable which references
>      SmalltalkImage current
>
> So I can do as well
>
>      SmalltalkImage current unloadAllKnownPackages
>
>
> Which I did
>
> Squeak4.2-10382-alpha.image went from 16998KB

Hm, I think Casey forgot to do [Smalltalk cleanUp: false] before saving
the image. Doing that saves 2.8MB.


Levente

>
> down to
> 10277kB
> (file sizes measured on Windows machine)
>
>>> So Long,
>>> -Tobias
>>>
>
>
>
>> Addendum:
>> It works.
>> resulting file size: 10411140, about 10,4 MB.
>> Nice.
>>
>> Btw, this script probably needs updating, The Help
>> browser, eg, does not get unloaded (same for TrueType)
>
> Yes, I tried it for the HelpSystem
>
> I added three more entries to the literal array in the method
>
> "Go unloading"
> #( 'ReleaseBuilder' 'ScriptLoader'
> '311Deprecated' '39Deprecated'
> 'Universes' 'SMLoader' 'SMBase' 'Installer-Core'
> 'VersionNumberTests' 'VersionNumber'
> 'Services-Base' 'PreferenceBrowser' 'Nebraska'
> 'ToolBuilder-MVC' 'ST80'
> 'CollectionsTests' 'GraphicsTests' 'KernelTests'  'MorphicTests'
> 'MultilingualTests' 'NetworkTests' 'ToolsTests' 'TraitsTests'
> 'SystemChangeNotification-Tests' 'FlexibleVocabularies'
> 'EToys' 'Protocols' 'XML-Parser' 'Tests' 'SUnitGUI',
> 'HelpSystem-Core', 'HelpSystem-Tests', 'HelpSystem-Project'
> ) do:[:pkgName| (MCPackage named: pkgName) unload].
>
>
> The image is now
> 10174kB (=100kB less10277kB).
>
> Probably not worth the effort at this time.
>
> Questions
> 1)  Which are the packages which are big and easy to unload.....?
> 2) Who does an attempt to unload the TrueType?
>
>
> --Hannes
>
>

Reply | Threaded
Open this post in threaded view
|

Re: About unloading of packages in the most recent Squeak 4.1 trunk

Casey Ransberger-2
Levente: totally. I mainly just wanted

a) image can start on Cog w/o segfaulting

b) doesn't need a long lengthy set of updates before being caught up with trunk

The cleanup methods will be run before we ship anyway:)

On Aug 23, 2010, at 4:29 AM, Levente Uzonyi <[hidden email]> wrote:

> On Mon, 23 Aug 2010, Hannes Hirzel wrote:
>
>> On 8/23/10, Tobias Pape <[hidden email]> wrote:
>>>
>>> Am 2010-08-23 um 11:14 schrieb Tobias Pape:
>>>
>>>> Hi,
>>>>
>>>> Am 2010-08-23 um 11:06 schrieb Hannes Hirzel:
>>>>
>>>>> and tried
>>>>>  SmalltalkImage unloadAllKnownPackages
>>>>
>>>> Do
>>>>    Smalltalk unloadAllKnownPackages
>>>>
>>>> instead, since Smalltalk is the sole instance
>>>> of SmalltalkImage.
>>
>> Thank you for point this out.
>>
>> In fact Smalltalk is the global variable which references
>>     SmalltalkImage current
>>
>> So I can do as well
>>
>>     SmalltalkImage current unloadAllKnownPackages
>>
>>
>> Which I did
>>
>> Squeak4.2-10382-alpha.image went from 16998KB
>
> Hm, I think Casey forgot to do [Smalltalk cleanUp: false] before saving the image. Doing that saves 2.8MB.
>
>
> Levente
>
>>
>> down to
>> 10277kB
>> (file sizes measured on Windows machine)
>>
>>>> So Long,
>>>>    -Tobias
>>>>
>>
>>
>>
>>> Addendum:
>>> It works.
>>>    resulting file size: 10411140, about 10,4 MB.
>>> Nice.
>>>
>>> Btw, this script probably needs updating, The Help
>>> browser, eg, does not get unloaded (same for TrueType)
>>
>> Yes, I tried it for the HelpSystem
>>
>> I added three more entries to the literal array in the method
>>
>> "Go unloading"
>>    #(    'ReleaseBuilder' 'ScriptLoader'
>>        '311Deprecated' '39Deprecated'
>>        'Universes' 'SMLoader' 'SMBase' 'Installer-Core'
>>        'VersionNumberTests' 'VersionNumber'
>>        'Services-Base' 'PreferenceBrowser' 'Nebraska'
>>        'ToolBuilder-MVC' 'ST80'
>>        'CollectionsTests' 'GraphicsTests' 'KernelTests'  'MorphicTests'
>>        'MultilingualTests' 'NetworkTests' 'ToolsTests' 'TraitsTests'
>>        'SystemChangeNotification-Tests' 'FlexibleVocabularies'
>>        'EToys' 'Protocols' 'XML-Parser' 'Tests' 'SUnitGUI',
>> 'HelpSystem-Core', 'HelpSystem-Tests', 'HelpSystem-Project'
>>    ) do:[:pkgName| (MCPackage named: pkgName) unload].
>>
>>
>> The image is now
>> 10174kB (=100kB less10277kB).
>>
>> Probably not worth the effort at this time.
>>
>> Questions
>> 1)  Which are the packages which are big and easy to unload.....?
>> 2) Who does an attempt to unload the TrueType?
>>
>>
>> --Hannes
>>
>>
>

Reply | Threaded
Open this post in threaded view
|

Re: About unloading of packages in the most recent Squeak 4.1 trunk

Tobias Pape
In reply to this post by Tobias Pape
>>
>> Smalltalk unloadAllKnownPackages
[…]

> Addendum:
> It works.
> resulting file size: 10411140, about 10,4 MB.
> Nice.
>
> Btw, this script probably needs updating, The Help
> browser, eg, does not get unloaded (same for TrueType)
>
> So Long,
> -Tobias
>

Addendum 2:

After unload, updating Trunk no longer works,
first error: trait no longer available.

just a note ;)

so long,
        -Tobias
Reply | Threaded
Open this post in threaded view
|

Re: About unloading of packages in the most recent Squeak 4.1 trunk

Edgar De Cleene



On 8/23/10 2:49 PM, "Tobias Pape" <[hidden email]> wrote:

> Addendum 2:
>
> After unload, updating Trunk no longer works,
> first error: trait no longer available.
>
> just a note ;)
>
> so long,
> -Tobias

It's why I modify the method.
Give a try to
http://ftp.squeak.org/Experiments/SqueakCore4.2-10382-alpha.zip

Edgar



Reply | Threaded
Open this post in threaded view
|

Re: About unloading of packages in the most recent Squeak 4.1 trunk

Casey Ransberger-2
In reply to this post by Tobias Pape
Yeah, which is why we currently have to work in the monolithic image. Once we've agreed on how we're going to load "community supported packages," we'll be able to move them out of the core image and out of the main trunk repository, which should solve this problem (I think, but I may not understand the issue: if I do, the problem is that we're trying to update things that aren't in the image after the unload.)

On Aug 23, 2010, at 10:49 AM, Tobias Pape <[hidden email]> wrote:

> After unload, updating Trunk no longer works,
> first error: trait no longer available.
>
> just a note ;)
>
> so long,
>    -Tobias

Reply | Threaded
Open this post in threaded view
|

Re: About unloading of packages in the most recent Squeak 4.1 trunk

Tobias Pape
In reply to this post by Casey Ransberger-2
Hi,
Am 2010-08-23 um 19:40 schrieb Casey Ransberger:

>> Hm, I think Casey forgot to do [Smalltalk cleanUp: false] before saving the image. Doing that saves 2.8MB.
>

makes no difference after unloading.
wich is somewhat surprising to me.

so long,
        -Tobias
Reply | Threaded
Open this post in threaded view
|

Re: About unloading of packages in the most recent Squeak 4.1 trunk

Levente Uzonyi-2
In reply to this post by Edgar De Cleene
On Mon, 23 Aug 2010, Edgar J. De Cleene wrote:

>
>
>
> On 8/23/10 2:49 PM, "Tobias Pape" <[hidden email]> wrote:
>
>> Addendum 2:
>>
>> After unload, updating Trunk no longer works,
>> first error: trait no longer available.
>>
>> just a note ;)
>>
>> so long,
>> -Tobias
>
> It's why I modify the method.
> Give a try to
> http://ftp.squeak.org/Experiments/SqueakCore4.2-10382-alpha.zip

I did, but it's not better at all. You simply didn't unload Traits, so
when the updater is trying to load 311-Deprecated it doesn't miss them.

The problem with updating after unloading is that the updater mechanism
has no information of package dependencies, so it will try to load
packages randomly which obviously won't work.


Levente

>
> Edgar
>
>
>
>

Reply | Threaded
Open this post in threaded view
|

Re: About unloading of packages in the most recent Squeak 4.1 trunk

Levente Uzonyi-2
In reply to this post by Casey Ransberger-2
On Mon, 23 Aug 2010, Casey Ransberger wrote:

> Levente: totally. I mainly just wanted
>
> a) image can start on Cog w/o segfaulting
>
> b) doesn't need a long lengthy set of updates before being caught up with trunk
>
> The cleanup methods will be run before we ship anyway:)

That's ok, but these prebuilt images could be a bit smaller this way.


Levente

>
> On Aug 23, 2010, at 4:29 AM, Levente Uzonyi <[hidden email]> wrote:
>
>> On Mon, 23 Aug 2010, Hannes Hirzel wrote:
>>
>>> On 8/23/10, Tobias Pape <[hidden email]> wrote:
>>>>
>>>> Am 2010-08-23 um 11:14 schrieb Tobias Pape:
>>>>
>>>>> Hi,
>>>>>
>>>>> Am 2010-08-23 um 11:06 schrieb Hannes Hirzel:
>>>>>
>>>>>> and tried
>>>>>>  SmalltalkImage unloadAllKnownPackages
>>>>>
>>>>> Do
>>>>>    Smalltalk unloadAllKnownPackages
>>>>>
>>>>> instead, since Smalltalk is the sole instance
>>>>> of SmalltalkImage.
>>>
>>> Thank you for point this out.
>>>
>>> In fact Smalltalk is the global variable which references
>>>     SmalltalkImage current
>>>
>>> So I can do as well
>>>
>>>     SmalltalkImage current unloadAllKnownPackages
>>>
>>>
>>> Which I did
>>>
>>> Squeak4.2-10382-alpha.image went from 16998KB
>>
>> Hm, I think Casey forgot to do [Smalltalk cleanUp: false] before saving the image. Doing that saves 2.8MB.
>>
>>
>> Levente
>>
>>>
>>> down to
>>> 10277kB
>>> (file sizes measured on Windows machine)
>>>
>>>>> So Long,
>>>>>    -Tobias
>>>>>
>>>
>>>
>>>
>>>> Addendum:
>>>> It works.
>>>>    resulting file size: 10411140, about 10,4 MB.
>>>> Nice.
>>>>
>>>> Btw, this script probably needs updating, The Help
>>>> browser, eg, does not get unloaded (same for TrueType)
>>>
>>> Yes, I tried it for the HelpSystem
>>>
>>> I added three more entries to the literal array in the method
>>>
>>> "Go unloading"
>>>    #(    'ReleaseBuilder' 'ScriptLoader'
>>>        '311Deprecated' '39Deprecated'
>>>        'Universes' 'SMLoader' 'SMBase' 'Installer-Core'
>>>        'VersionNumberTests' 'VersionNumber'
>>>        'Services-Base' 'PreferenceBrowser' 'Nebraska'
>>>        'ToolBuilder-MVC' 'ST80'
>>>        'CollectionsTests' 'GraphicsTests' 'KernelTests'  'MorphicTests'
>>>        'MultilingualTests' 'NetworkTests' 'ToolsTests' 'TraitsTests'
>>>        'SystemChangeNotification-Tests' 'FlexibleVocabularies'
>>>        'EToys' 'Protocols' 'XML-Parser' 'Tests' 'SUnitGUI',
>>> 'HelpSystem-Core', 'HelpSystem-Tests', 'HelpSystem-Project'
>>>    ) do:[:pkgName| (MCPackage named: pkgName) unload].
>>>
>>>
>>> The image is now
>>> 10174kB (=100kB less10277kB).
>>>
>>> Probably not worth the effort at this time.
>>>
>>> Questions
>>> 1)  Which are the packages which are big and easy to unload.....?
>>> 2) Who does an attempt to unload the TrueType?
>>>
>>>
>>> --Hannes
>>>
>>>
>>
>
>

Reply | Threaded
Open this post in threaded view
|

Re: About unloading of packages in the most recent Squeak 4.1 trunk

Levente Uzonyi-2
In reply to this post by Tobias Pape
On Mon, 23 Aug 2010, Tobias Pape wrote:

> Hi,
> Am 2010-08-23 um 19:40 schrieb Casey Ransberger:
>
>>> Hm, I think Casey forgot to do [Smalltalk cleanUp: false] before saving the image. Doing that saves 2.8MB.
>>
>
> makes no difference after unloading.
> wich is somewhat surprising to me.

But it's ok. [Smalltalk cleanUp: false] just releases cached objects.
Some of these objects are released when the packages are unloaded, others
are cleaned by SmalltalkImage >> #unloadAllKnownPackages, like:
  Undeclared removeUnreferencedKeys.
  StandardScriptingSystem initialize.
  MCFileBasedRepository flushAllCaches.
  MCDefinition clearInstances.
  Behavior flushObsoleteSubclasses.
  ChangeSet current clear.
etc.


Levente

>
> so long,
> -Tobias
>

Reply | Threaded
Open this post in threaded view
|

Re: About unloading of packages in the most recent Squeak 4.1 trunk

Bert Freudenberg
In reply to this post by Levente Uzonyi-2
On 24.08.2010, at 00:03, Levente Uzonyi wrote:

> The problem with updating after unloading is that the updater mechanism has no information of package dependencies, so it will try to load packages randomly which obviously won't work.

It does not load packages randomly, but in the order defined in the update config map.

Since updating should not bring in the unloaded packages we would need to inform the updater to skip them.

- Bert -


Reply | Threaded
Open this post in threaded view
|

Re: About unloading of packages in the most recent Squeak 4.1 trunk

Levente Uzonyi-2
On Tue, 24 Aug 2010, Bert Freudenberg wrote:

> On 24.08.2010, at 00:03, Levente Uzonyi wrote:
>
>> The problem with updating after unloading is that the updater mechanism has no information of package dependencies, so it will try to load packages randomly which obviously won't work.
>
> It does not load packages randomly, but in the order defined in the update config map.

That's right, but it's still random from the package dependency POV.

>
> Since updating should not bring in the unloaded packages we would need to inform the updater to skip them.

Right.


Levente

>
> - Bert -
>
>
>

Reply | Threaded
Open this post in threaded view
|

Re: About unloading of packages in the most recent Squeak 4.1 trunk

Bert Freudenberg

On 24.08.2010, at 00:34, Levente Uzonyi wrote:

> On Tue, 24 Aug 2010, Bert Freudenberg wrote:
>
>> On 24.08.2010, at 00:03, Levente Uzonyi wrote:
>>
>>> The problem with updating after unloading is that the updater mechanism has no information of package dependencies, so it will try to load packages randomly which obviously won't work.
>>
>> It does not load packages randomly, but in the order defined in the update config map.
>
> That's right, but it's still random from the package dependency POV.

What makes you think so? The order is chosen (manually) so that dependent packages are loaded before depending packages.

- Bert -

>> Since updating should not bring in the unloaded packages we would need to inform the updater to skip them.
>
> Right.
>
>
> Levente
>
>>
>> - Bert -
>>
>>
>>
>



Reply | Threaded
Open this post in threaded view
|

Re: About unloading of packages in the most recent Squeak 4.1 trunk

Levente Uzonyi-2
On Tue, 24 Aug 2010, Bert Freudenberg wrote:

>
> On 24.08.2010, at 00:34, Levente Uzonyi wrote:
>
>> On Tue, 24 Aug 2010, Bert Freudenberg wrote:
>>
>>> On 24.08.2010, at 00:03, Levente Uzonyi wrote:
>>>
>>>> The problem with updating after unloading is that the updater mechanism has no information of package dependencies, so it will try to load packages randomly which obviously won't work.
>>>
>>> It does not load packages randomly, but in the order defined in the update config map.
>>
>> That's right, but it's still random from the package dependency POV.
>
> What makes you think so? The order is chosen (manually) so that dependent packages are loaded before depending packages.

If that would be the case, then updating an "unloaded" image would reload
all unloaded packages. But that doesn't work. For example 311Deprecated
depends on Traits, but Traits is at the end of the list while
311Deprecated is the first non-dummy package.


Levente

>
> - Bert -
>
>>> Since updating should not bring in the unloaded packages we would need to inform the updater to skip them.
>>
>> Right.
>>
>>
>> Levente
>>
>>>
>>> - Bert -
>>>
>>>
>>>
>>
>
>
>
>

123