Login  Register

Re: [Pharo-dev] Large images reasons [WAS] Re: Pharo 2.0 with Seaside + DBXTalk + GlorpDBX + Magritte 3 + TWBS is getting slower and slower

Posted by Goubier Thierry on Aug 01, 2013; 7:37am
URL: https://forum.world.st/Large-images-reasons-WAS-Re-Pharo-2-0-with-Seaside-DBXTalk-GlorpDBX-Magritte-3-TWBS-is-getting-slower-tp4701743p4701786.html

Hi Stef,

With Dale, we're working on a FileTree / GitFileTree without ancestors
info in the repository [1] :) ...

... but recreated on the fly by GitFileTree for the Monticello history
browsing tools [2] :(

[1] is underway and ...
[2] is already working in
https://github.com/dalehenrich/filetree/tree/pharo2.0/repository/MonticelloFileTree-Git.package

Thierry

Le 01/08/2013 08:56, Stéphane Ducasse a écrit :
> since years marcus is telling that MC storing ancestor information is
> doomed but we do not have something to really replace it.
>
> Stef


> On Jul 31, 2013, at 8:53 PM, Mariano Martinez Peck
> <[hidden email] <mailto:[hidden email]>> wrote:
>
>> OK, this was was my experiment....
>>
>> Image fresh with all my app and dependencies loades: 30MB
>>
>> After using it for some days/weeks: 160MB.
>>
>> SpaceTally new printSpaceAnalysis showed:
>>
>> Class                                          code space # instances
>>  inst space     percent   inst average size
>> ByteString                                           2785      413144
>>     116244078       69.90              281.36
>> Array                                                3712      181772
>>       8466668        5.10               46.58
>> ByteArray                                            8574        1319
>>       8186802        4.90             6206.82
>> Bitmap                                               3653         303
>>       6656340        4.00            21968.12
>> CompiledMethod                                      22467       90554
>>       5685374        3.40               62.78
>>
>>
>> After executing ImageCleaner cleanUpForRelease: 36MB
>>
>> Then...I searched which part of #cleanUpForRelease: was making the
>> difference, and it was:
>>
>> Smalltalk cleanUp: true except: #() confirming: false.
>>
>> So now it was time to know WHICH class did the diference, so I modified
>> #cleanUp: aggressive except: exclusions confirming: aBool
>>
>> in these lines:
>>
>> "Run the cleanup code"
>> classes
>> do:[:aClass|
>> Transcript show: 'Image size before cleaning ', aClass name, ' : ',
>> Smalltalk imagePath asFileReference size asString.
>> aClass cleanUp: aggressive.
>> 3 timesRepeat: [Smalltalk garbageCollect].
>> Smalltalk snapshot: true andQuit: false.
>> Transcript show: 'Image size after cleaning ', aClass name, ' : ',
>> Smalltalk imagePath asFileReference size asString.
>> ]
>> displayingProgress: [:aClass| 'Cleaning up in ', aClass name].
>> I then opened a Transcript, and evaluated
>>
>> Smalltalk cleanUp: true except: #() confirming: false.
>>
>> I went to prepare Mate, and when I come back, the result was, of course:
>>
>> "Image size after cleaning MCFileBasedRepository : 39744008"
>>
>> That clean up ends up doing:
>>
>> flushAllCaches
>> self allSubInstancesDo: [:ea | ea flushCache]
>> So it sends #flushCache to all instances of MCHttpRepository and
>> MCFileBasedRepository.
>>
>> Now what I wanted to see if it there was a particular repo that could
>> take most of the space (like package-cache).
>> And indeed, it was...I modified #flushCaches to:
>>
>> flushAllCaches
>> | file |
>> file := 'repos.txt' asFileReference writeStream text.
>> self allSubInstancesDo: [:each |
>> file nextPutAll: 'Image size before cleaning ', each printString, ' :
>> ', Smalltalk imagePath asFileReference size asString; cr.
>> each flushCache.
>> 3 timesRepeat: [Smalltalk garbageCollect].
>> Smalltalk snapshot: true andQuit: false.
>> file nextPutAll: 'Image size after cleaning ', each printString, ' :
>> ', Smalltalk imagePath asFileReference size asString;cr.
>> ].
>> file flush; close.
>> And then I looked in the 'repos.txt' file. My package cache repo
>> cleaned 60 MB. Glorp cleaned 35MB. Seaside30 cleaned 10MB.
>> So...cleaning cache of just 3 repos frees approx 100MB.
>>
>> The question is....can we flush the cache safely? If they are called
>> "cache", then I guess yes, we can.
>>
>> Thoughts?
>>
>> Thanks,
>>
>>
>>
>> On Wed, Jul 31, 2013 at 10:48 AM, Mariano Martinez Peck
>> <[hidden email] <mailto:[hidden email]>> wrote:
>>
>>     Guys, I have images also with seaside, magritte, glorp,
>>     postgresV2, etc and it is also around 200MB.
>>     I will try to do some research today and let you know.
>>
>>     Cheers,
>>
>>
>>     On Tue, Jul 30, 2013 at 8:55 AM, Marcus Denker
>>     <[hidden email] <mailto:[hidden email]>> wrote:
>>
>>
>>         On Jul 30, 2013, at 1:49 PM, "[hidden email]
>>         <mailto:[hidden email]>" <[hidden email]
>>         <mailto:[hidden email]>> wrote:
>>
>>         > the changes file contained passwords and I replaced the
>>         text.  So offsets may be wrong due to that.
>>         >
>>         Yes, the first thing I wanted to do is to recompile
>>         everything. Does not work.
>>
>>         > Memorymonitor is not doing fanct stuff. It just counts
>>         instances.
>>         >
>>         Yes, but maybe it holds on to these instances?
>>
>>                 Marcus
>>
>>
>>
>>
>>
>>     --
>>     Mariano
>>     http://marianopeck.wordpress.com <http://marianopeck.wordpress.com/>
>>
>>
>>
>>
>> --
>> Mariano
>> http://marianopeck.wordpress.com <http://marianopeck.wordpress.com/>
>

--
Thierry Goubier
CEA list
Laboratoire des Fondations des Systèmes Temps Réel Embarqués
91191 Gif sur Yvette Cedex
France
Phone/Fax: +33 (0) 1 69 08 32 92 / 83 95