Ballooning image size! Approaching 400MB
Locked
Ballooning image size! Approaching 400MB
 
|
My image size keeps growing and growing. I've been scouring Google and mailing lists, trying to find a solution.
Observables:
Somehow, these instances I create are not 'releasing'. All of my Classes are direct subclasses of Object. Interestingly enough, all these objects sit in a single collection for the high-level model - and there are only 67 instances of that Object... here's the hierarchy: MyTunes - instance holds a Dictionary of Libraries, rootFolder and tags collection
Here's my start code in my app... (I don't know how to use SpApplication, SpPresenter and my model(s) together - this may be part of the problem)..
While I'd love help with my App/Pres/Model - my first goal is to stop the rapid growth of my object. Questions? Thanks! Russ Whaley (USA, Ohio) Russ Whaley [hidden email] |
Re: Ballooning image size! Approaching 400MB
|
|
Hi Russ,
It is quite hard to debug somebody else's application, especially without access to the code, but here are some suggestions of what I would do. Can you move your code to a new/fresh image and see what happens there (especially after just one iteration of loading your data) ? Is there a difference between loading your data and opening your app with respect to the retention of garbage ? This would help to determine if this is a UI problem or a domain model problem. There are tools that help you to figure out who holds a specific object ('Reverse pointers to' in the Inspector) but these are not so easy to use (you get lots of data). Sven > On 31 Mar 2020, at 16:55, Russ Whaley <[hidden email]> wrote: > > My image size keeps growing and growing. I've been scouring Google and mailing lists, trying to find a solution. > > Observables: > • Running this 'query' shows I have a whole lot instances of Objects I've created that are not "releasing". There are 65000 instances of one Object/Class, and 350,000 of another. (there are more, these are just the big hitters). > (Object allSubclasses collect: [ :aClass | > aClass -> aClass allInstances size]) > sort: [ :a :b | a value > b value ]. > • Each time I load my app/UI - which creates/loads another 5000 of one object and 25,000 of the 2nd object - my Class>>instanceCount(s) keep growing and so does my image size. > • I've done: > • Smalltalk garbageCollect. > • Smalltalk garbageCollectMost. > • 10 timesRepeat: [ Smalltalk garbageCollect]. (many, many times) > Somehow, these instances I create are not 'releasing'. All of my Classes are direct subclasses of Object. Interestingly enough, all these objects sit in a single collection for the high-level model - and there are only 67 instances of that Object... here's the hierarchy: > > MyTunes - instance holds a Dictionary of Libraries, rootFolder and tags collection > Library - instance(s) hold Dictionary of Artists & Playlists, tags collection, and MyTunes reference > Artist - instances hold Dictionary of Albums, and Library reference > Album - instances hold Dictionary of Tracks, and Artist reference > Track - instances hold TagInfo, Album reference > TagInfo - instances hold fileReference and ByteString of ID3 tag info. > PlayList - instances hold Dictionary of Tracks, Library reference. > > Here's my start code in my app... (I don't know how to use SpApplication, SpPresenter and my model(s) together - this may be part of the problem).. > > start > "Override this to start your app" > "*** I don't know how to use SpApplicatoin, SpPresenter and my model MyTunes ***" > > | myTunes tmpLibrary myPres | > > "where do I store the model -> MyTunes?" > myTunes := MyTunes new. > > "this is just for testing, but the actual re-load of the library takes 15-20 minutes (60,000 files!)" > tmpLibrary := myTunes readLibraryFromSTONfile. > myTunes addLibrary: tmpLibrary. > > "where do I store the presenter -> MyTunesPresenter? " > myPres := MyTunesPresenter on: tmpLibrary. > myPres openWithSpec. > > While I'd love help with my App/Pres/Model - my first goal is to stop the rapid growth of my object. > > Questions? > > Thanks! > Russ Whaley (USA, Ohio) > > -- > Russ Whaley > [hidden email] |
Re: Ballooning image size! Approaching 400MB
|
|
In reply to this post by Russ Whaley
Ciao,
> My image size keeps growing and growing. I've been scouring Google and mailing lists, trying to find a solution. i have the same problem with a Pharo 7 image. Maybe it has nothing to do. But after close all the class browser windows the image save return to a " valid " size. Dario |
Re: Ballooning image size! Approaching 400MB
|
|
In reply to this post by Sven Van Caekenberghe-2
Sven, Thanks for the reply :) I will try to move my code over to another image (I'm not very good with GitHub - we'll see how this goes). But it just seems like there ought to be a way to at least delete all of these instances - as they are pretty easy to get ahold of (Class allInstancesDo:). But I can't find a way to delete them. I can count them, I can display content - attached is a screenshot showing for each Album instance, I can display it and data from it's embedded Artist and collection of Tracks. Attached are a couple of screen shots of the 'query' I mentioned in my previous note... as you can see - Tracks, Albums, (FileReference is my fault, also - I use those to hand music files to my objects for processing. Those are hanging around as well). On Tue, Mar 31, 2020 at 11:13 AM Sven Van Caekenberghe <[hidden email]> wrote: Hi Russ, Russ Whaley [hidden email]  2020-03-31 12.18.45 pm.png (252K) Download Attachment 2020-03-31 12.07.58 pm.png (290K) Download Attachment 2020-03-31 12.02.44 pm.png (202K) Download Attachment |
Re: Ballooning image size! Approaching 400MB
Administrator
|
On Tue, Mar 31, 2020 at 9:25 AM Russ Whaley <[hidden email]> wrote:
Russ, this kind of thing is a common occurrence in Smalltalk
GUI applications. In my experience, it is often dangling dependencies resulting in things being held. The reference in question might be to the model itself or to some particular subset which also happens to include a back reference to the whole model. These dependencies typically are installed as event handlers in the GUI. It's common to establish a dependency but fail to release it. In your first message, you listed the major classes of your model. Get the object counts for each. Close the application UI (not Pharo) and re-open it. Run the counts again. You will probably see the root model instance count increase by one. I don't work with Pharo, so I cannot give you specific guidance for tracking these down. I'm certain others can explain GUI dependencies and dangling references in the Pharo environment, to give you more focussed guidance.
|
Re: Ballooning image size! Approaching 400MB
|
|
In reply to this post by Trussardi Dario Romano
Dario, Thanks for the reply! When I closed my image - after running that huge query and leaving it in an inspector... my image grew to 1.3GB, lol. When I closed the inspector, and all the class browsers, did the garbage collection, etc., it dropped back to 384MB. Still huge, and all my Class instances are still there. I'm getting ready to also drop my Playground - after I copy all the code snippets out and see if that has any impact. I'm also going to try a fresh image and see if there is the same growth pattern as here. Perhaps I'm doing something wrong with the way I'm creating and storing my object instances - which is generally Class new, fill out some attributes, then add it to a collection or Dictionary. All my Object tests create sample data (objects) - over and over as well. On Tue, Mar 31, 2020 at 12:20 PM dario.trussardi65 <[hidden email]> wrote: Ciao, Russ Whaley [hidden email] |
Re: Ballooning image size! Approaching 400MB
|
|
Here is some additional data - attached - I checked the number of instances on select Classes before a UI load, during the UI load and after the UI load (and again). The class instances grow as expected, but do no release. I'm doing something wrong. I'm going to try a fresh image next, however, I expect the same thing to happen, just starting over at zero. On Tue, Mar 31, 2020 at 12:57 PM Russ Whaley <[hidden email]> wrote:
Russ Whaley [hidden email] |
Re: Ballooning image size! Approaching 400MB
|
|
Hi Russ - a quick look at your stats seems to indicate that something is holding on to your model, I didn’t understand the first object growing by 2, but the second increases by 1 each time, and it looks like some form of root object holding on to the others ? This of course means the GC can’t reclaim the chain of objects if something is holding on to that root globally. Assigning something in the playground will hold onto it (I believe) so you can continue to reuse and inspect it - so that’s expected, but this looks like something in your UI? You can break the chain manually (but it’s not a full solution), simply iterate over your root and become: String new (Save your image before doing it and see if it works to free up the cycles). Eg try something like this (haven’t tried this in Pharo myself, but worked in VA, so it should work here) MyRootModel allInstances do: [ :m | m become: String new ] If this works and reduces your memory usage, you now need to inspect one of those MyRootModel instances and see who is referencing it, and explain why. This is Sven’s pointerTo explanation. For roots, you sometimes can use a WeakDictionary so that when nothing is referencing them, they get gc’d if you are doing some caching or have some factory concept. It’s not uncommon to hit this when you get your system nearing completion , it’s the downside of not having to worry about memory referencing - ultimately you have to worry about it at the edges. It is possible there is a Pharo bug, as over time I see large Pharo images but I was just messing around and put it down to failed experiments. Hope this helps. Tim On 31 Mar 2020, at 20:47, Russ Whaley <[hidden email]> wrote:
|
|
|
Tim,
out of curiosity: why do you suggest to
create hundreds of thousands of Strings instead of become: nil?
Does Pharo do two-way become: ?
I'be been nilling instances for a little over 2 decades in VAST so far and never had any troubles with it... Other than that: the become: nil (or
String new as you suggest) thing was the first that came to my
mind for the intermediate solution as well. Of course that doesn't
fix the code which doesn't dispose of the objects in the UI or
model...
Joachim
Am 01.04.20 um 08:42 schrieb Tim
Mackinnon:
-- ----------------------------------------------------------------------- Objektfabrik Joachim Tuchel [hidden email] Fliederweg 1 http://www.objektfabrik.de D-71640 Ludwigsburg http://joachimtuchel.wordpress.com Telefon: +49 7141 56 10 86 0 Fax: +49 7141 56 10 86 1 |
Re: Ballooning image size! Approaching 400MB
|
|
Yeah, I was taught (and it may be out of date), that as become: is a 2 way operation the identity of nil (as a singleton) would get swizzled, so it was safer to create a simple new object that could be easily gc’d (that’s what I vaguely recall anyway). (This was in early VA from the OTI guys, but maybe that gets handled better in modern VA, as I think become: is two way in VA right?)
Tim
|
|
|
Tim,
okay, so I understand the reason.
Thanks for clarifying/confirming?
I don't think VAST has two-way become.
IIRC, VW has. At least I can tell you using nil instead of String
new has worked for me on VAST versions ever since 3.0 in the early
nineties.
What about Pharo? Does it do one-way or
two-way become?
Joachim
Am 01.04.20 um 10:29 schrieb Tim
Mackinnon:
Yeah, I was taught (and it may be out of date), that as become: is a 2 way operation the identity of nil (as a singleton) would get swizzled, so it was safer to create a simple new object that could be easily gc’d (that’s what I vaguely recall anyway). (This was in early VA from the OTI guys, but maybe that gets handled better in modern VA, as I think become: is two way in VA right?)
-- ----------------------------------------------------------------------- Objektfabrik Joachim Tuchel [hidden email] Fliederweg 1 http://www.objektfabrik.de D-71640 Ludwigsburg http://joachimtuchel.wordpress.com Telefon: +49 7141 56 10 86 0 Fax: +49 7141 56 10 86 1 |
Re: Ballooning image size! Approaching 400MB
|
|
ProtoObject>>become: is two-way in Pharo.
"...Swap the object pointers of the receiver and the argument. All variables in the entire system that used to point to the receiver now point to the argument, and vice-versa." In the unlikely event that I have understood correctly, Array>>elementsForwardIdentityTo: is Pharo's one-way become. On Wed, 1 Apr 2020 at 22:43, [hidden email] <[hidden email]> wrote: > > Tim, > > okay, so I understand the reason. Thanks for clarifying/confirming? > > I don't think VAST has two-way become. IIRC, VW has. At least I can tell you using nil instead of String new has worked for me on VAST versions ever since 3.0 in the early nineties. > > What about Pharo? Does it do one-way or two-way become? > > > Joachim > > > > Am 01.04.20 um 10:29 schrieb Tim Mackinnon: > > Yeah, I was taught (and it may be out of date), that as become: is a 2 way operation the identity of nil (as a singleton) would get swizzled, so it was safer to create a simple new object that could be easily gc’d (that’s what I vaguely recall anyway). (This was in early VA from the OTI guys, but maybe that gets handled better in modern VA, as I think become: is two way in VA right?) > > Tim > > On 1 Apr 2020, at 08:11, [hidden email] wrote: > > Tim, > > out of curiosity: why do you suggest to create hundreds of thousands of Strings instead of become: nil? > > Does Pharo do two-way become: ? > I'be been nilling instances for a little over 2 decades in VAST so far and never had any troubles with it... > > Other than that: the become: nil (or String new as you suggest) thing was the first that came to my mind for the intermediate solution as well. Of course that doesn't fix the code which doesn't dispose of the objects in the UI or model... > > > Joachim > > > > > Am 01.04.20 um 08:42 schrieb Tim Mackinnon: > > Hi Russ - a quick look at your stats seems to indicate that something is holding on to your model, I didn’t understand the first object growing by 2, but the second increases by 1 each time, and it looks like some form of root object holding on to the others ? This of course means the GC can’t reclaim the chain of objects if something is holding on to that root globally. > > Assigning something in the playground will hold onto it (I believe) so you can continue to reuse and inspect it - so that’s expected, but this looks like something in your UI? > > You can break the chain manually (but it’s not a full solution), simply iterate over your root and become: String new (Save your image before doing it and see if it works to free up the cycles). Eg try something like this (haven’t tried this in Pharo myself, but worked in VA, so it should work here) > > MyRootModel allInstances do: [ :m | m become: String new ] > > If this works and reduces your memory usage, you now need to inspect one of those MyRootModel instances and see who is referencing it, and explain why. This is Sven’s pointerTo explanation. > > For roots, you sometimes can use a WeakDictionary so that when nothing is referencing them, they get gc’d if you are doing some caching or have some factory concept. > > It’s not uncommon to hit this when you get your system nearing completion , it’s the downside of not having to worry about memory referencing - ultimately you have to worry about it at the edges. > > It is possible there is a Pharo bug, as over time I see large Pharo images but I was just messing around and put it down to failed experiments. > > Hope this helps. > > Tim > > On 31 Mar 2020, at 20:47, Russ Whaley <[hidden email]> wrote: > > > Here is some additional data - attached - I checked the number of instances on select Classes before a UI load, during the UI load and after the UI load (and again). The class instances grow as expected, but do no release. I'm doing something wrong. I'm going to try a fresh image next, however, I expect the same thing to happen, just starting over at zero. > > > > On Tue, Mar 31, 2020 at 12:57 PM Russ Whaley <[hidden email]> wrote: >> >> Dario, >> Thanks for the reply! >> >> When I closed my image - after running that huge query and leaving it in an inspector... my image grew to 1.3GB, lol. >> >> When I closed the inspector, and all the class browsers, did the garbage collection, etc., it dropped back to 384MB. Still huge, and all my Class instances are still there. I'm getting ready to also drop my Playground - after I copy all the code snippets out and see if that has any impact. I'm also going to try a fresh image and see if there is the same growth pattern as here. >> >> Perhaps I'm doing something wrong with the way I'm creating and storing my object instances - which is generally Class new, fill out some attributes, then add it to a collection or Dictionary. >> >> All my Object tests create sample data (objects) - over and over as well. >> >> >> On Tue, Mar 31, 2020 at 12:20 PM dario.trussardi65 <[hidden email]> wrote: >>> >>> Ciao, >>> >>> >>> > My image size keeps growing and growing. I've been scouring Google and mailing lists, trying to find a solution. >>> >>> i have the same problem with a Pharo 7 image. >>> >>> Maybe it has nothing to do. >>> >>> But after close all the class browser windows the image save return to a " valid " size. >>> >>> Dario >> >> >> >> -- >> Russ Whaley >> [hidden email] > > > > -- > Russ Whaley > [hidden email] > <Class instances tests.pdf> > > > -- > ----------------------------------------------------------------------- > Objektfabrik Joachim Tuchel mailto:[hidden email] > Fliederweg 1 http://www.objektfabrik.de > D-71640 Ludwigsburg http://joachimtuchel.wordpress.com > Telefon: +49 7141 56 10 86 0 Fax: +49 7141 56 10 86 1 > > > > > -- > ----------------------------------------------------------------------- > Objektfabrik Joachim Tuchel mailto:[hidden email] > Fliederweg 1 http://www.objektfabrik.de > D-71640 Ludwigsburg http://joachimtuchel.wordpress.com > Telefon: +49 7141 56 10 86 0 Fax: +49 7141 56 10 86 1 > > |
Re: Ballooning image size! Approaching 400MB
|
|
> Am 01.04.2020 um 12:17 schrieb Richard O'Keefe <[hidden email]>: > > ProtoObject>>become: is two-way in Pharo. > "...Swap the object pointers of the receiver and the argument. > All variables in the entire system that used to point to the > receiver now point to the argument, and vice-versa." > > In the unlikely event that I have understood correctly, > Array>>elementsForwardIdentityTo: > is Pharo's one-way become. > Norbert > On Wed, 1 Apr 2020 at 22:43, [hidden email] > <[hidden email]> wrote: >> >> Tim, >> >> okay, so I understand the reason. Thanks for clarifying/confirming? >> >> I don't think VAST has two-way become. IIRC, VW has. At least I can tell you using nil instead of String new has worked for me on VAST versions ever since 3.0 in the early nineties. >> >> What about Pharo? Does it do one-way or two-way become? >> >> >> Joachim >> >> >> >> Am 01.04.20 um 10:29 schrieb Tim Mackinnon: >> >> Yeah, I was taught (and it may be out of date), that as become: is a 2 way operation the identity of nil (as a singleton) would get swizzled, so it was safer to create a simple new object that could be easily gc’d (that’s what I vaguely recall anyway). (This was in early VA from the OTI guys, but maybe that gets handled better in modern VA, as I think become: is two way in VA right?) >> >> Tim >> >> On 1 Apr 2020, at 08:11, [hidden email] wrote: >> >> Tim, >> >> out of curiosity: why do you suggest to create hundreds of thousands of Strings instead of become: nil? >> >> Does Pharo do two-way become: ? >> I'be been nilling instances for a little over 2 decades in VAST so far and never had any troubles with it... >> >> Other than that: the become: nil (or String new as you suggest) thing was the first that came to my mind for the intermediate solution as well. Of course that doesn't fix the code which doesn't dispose of the objects in the UI or model... >> >> >> Joachim >> >> >> >> >> Am 01.04.20 um 08:42 schrieb Tim Mackinnon: >> >> Hi Russ - a quick look at your stats seems to indicate that something is holding on to your model, I didn’t understand the first object growing by 2, but the second increases by 1 each time, and it looks like some form of root object holding on to the others ? This of course means the GC can’t reclaim the chain of objects if something is holding on to that root globally. >> >> Assigning something in the playground will hold onto it (I believe) so you can continue to reuse and inspect it - so that’s expected, but this looks like something in your UI? >> >> You can break the chain manually (but it’s not a full solution), simply iterate over your root and become: String new (Save your image before doing it and see if it works to free up the cycles). Eg try something like this (haven’t tried this in Pharo myself, but worked in VA, so it should work here) >> >> MyRootModel allInstances do: [ :m | m become: String new ] >> >> If this works and reduces your memory usage, you now need to inspect one of those MyRootModel instances and see who is referencing it, and explain why. This is Sven’s pointerTo explanation. >> >> For roots, you sometimes can use a WeakDictionary so that when nothing is referencing them, they get gc’d if you are doing some caching or have some factory concept. >> >> It’s not uncommon to hit this when you get your system nearing completion , it’s the downside of not having to worry about memory referencing - ultimately you have to worry about it at the edges. >> >> It is possible there is a Pharo bug, as over time I see large Pharo images but I was just messing around and put it down to failed experiments. >> >> Hope this helps. >> >> Tim >> >> On 31 Mar 2020, at 20:47, Russ Whaley <[hidden email]> wrote: >> >> >> Here is some additional data - attached - I checked the number of instances on select Classes before a UI load, during the UI load and after the UI load (and again). The class instances grow as expected, but do no release. I'm doing something wrong. I'm going to try a fresh image next, however, I expect the same thing to happen, just starting over at zero. >> >> >> >> On Tue, Mar 31, 2020 at 12:57 PM Russ Whaley <[hidden email]> wrote: >>> >>> Dario, >>> Thanks for the reply! >>> >>> When I closed my image - after running that huge query and leaving it in an inspector... my image grew to 1.3GB, lol. >>> >>> When I closed the inspector, and all the class browsers, did the garbage collection, etc., it dropped back to 384MB. Still huge, and all my Class instances are still there. I'm getting ready to also drop my Playground - after I copy all the code snippets out and see if that has any impact. I'm also going to try a fresh image and see if there is the same growth pattern as here. >>> >>> Perhaps I'm doing something wrong with the way I'm creating and storing my object instances - which is generally Class new, fill out some attributes, then add it to a collection or Dictionary. >>> >>> All my Object tests create sample data (objects) - over and over as well. >>> >>> >>> On Tue, Mar 31, 2020 at 12:20 PM dario.trussardi65 <[hidden email]> wrote: >>>> >>>> Ciao, >>>> >>>> >>>>> My image size keeps growing and growing. I've been scouring Google and mailing lists, trying to find a solution. >>>> >>>> i have the same problem with a Pharo 7 image. >>>> >>>> Maybe it has nothing to do. >>>> >>>> But after close all the class browser windows the image save return to a " valid " size. >>>> >>>> Dario >>> >>> >>> >>> -- >>> Russ Whaley >>> [hidden email] >> >> >> >> -- >> Russ Whaley >> [hidden email] >> <Class instances tests.pdf> >> >> >> -- >> ----------------------------------------------------------------------- >> Objektfabrik Joachim Tuchel mailto:[hidden email] >> Fliederweg 1 http://www.objektfabrik.de >> D-71640 Ludwigsburg http://joachimtuchel.wordpress.com >> Telefon: +49 7141 56 10 86 0 Fax: +49 7141 56 10 86 1 >> >> >> >> >> -- >> ----------------------------------------------------------------------- >> Objektfabrik Joachim Tuchel mailto:[hidden email] >> Fliederweg 1 http://www.objektfabrik.de >> D-71640 Ludwigsburg http://joachimtuchel.wordpress.com >> Telefon: +49 7141 56 10 86 0 Fax: +49 7141 56 10 86 1 >> >> > |
Re: Ballooning image size! Approaching 400MB
|
Administrator
|
In reply to this post by Tim Mackinnon
On Wed, Apr 1, 2020, 01:30 Tim Mackinnon <[hidden email]> wrote:
VA become: is one way. You can use multiBecome: to have a two way become:. I don't have the details to hand.
|
Re: Ballooning image size! Approaching 400MB
|
|
In reply to this post by Tim Mackinnon
Tim, Thanks for the reply! When I eliminated (with your become: solution), it did not a small chunk of the objects instances off. Another 'holder' of object instances accounted for another slug. But things still were not dropping like I hoped (this is without just setting everything to strings - but doing so systematically by object to perhaps see what was holding on to things). Finally I got things down to a few objects with instances in the hundreds and thousands - instead of 50,000 and 400,000 - and I finally set everything to String new. Interestingly enough, after garbage collection, when I do String instanceCount - I get 0 (zero) instances. All that - and thank you again - I closed all the system browsers, checked playground to delete bindings, I did a bunch of garbageCollection runs, did a System>Do Image Cleanup... then did a Save As. And... it is 8MB bigger than before I did all this cleanup. My image was 346MB and grew to 354MB AFTER the cleanup. Go figure. Maybe it will automagically GC sometime. I'm looking into what you and Sven mentioned - the pointersTo - I'm trying to figure out how to use it. A couple of basic questions...
I'm running some tests of the GUI after clearing out all the Object instances... and I'm seeing the growth as expected... it just doesn't go away when I close the GUI window. I'm not using SpApp correctly, nor am I using SpPresenter correctly (I'm forcing my model data into an SpPresenter instVar (via the SpApp start) so the presenter can use my data. This is all wrong but I'm struggling finding good examples in Spec2 of how to get these to work together. Thanks for the info - your suggestions have been very helpful! Russ On Wed, Apr 1, 2020 at 2:43 AM Tim Mackinnon <[hidden email]> wrote:
Russ Whaley [hidden email] |
Re: Ballooning image size! Approaching 400MB
|
|
Okay! I made some additional changes to my app/presenter/model so that I'm doing this: MyApp>>start
I changed my code in Presenter to reference the model - and it pops right up! When I close the GUI, GC and check for errant object instances... NONE!! I can open the UI over and over and my object instances stay at zero. So it feels like I'm on the right track. I still need a bunch of work/learning around the App/Presenter/Model interactions... so if anyone can point me to a good tutorial on this, including changing data and UI widgets - I'd greatly appreciate it. Thanks to all on this issue! Russ On Wed, Apr 1, 2020 at 12:14 PM Russ Whaley <[hidden email]> wrote:
Russ Whaley [hidden email] |
Re: Ballooning image size! Approaching 400MB
|
|
Hi Russ,
That is great to hear. I wanted to add that Pharo has automatic memory management and there is normally no need for doing anything manually (like nilling out instance variables let alone ugly become hacks). Your model looks pretty harmless, even with cycles there should be no problems. All this does not mean that you cannot (by accident maybe) make mistakes and keep global references to your larger model that prevents it from being garbage collected. Typically, fixing the one culprit solves all your problems. The trick is finding it ;-) Sven > On 1 Apr 2020, at 20:04, Russ Whaley <[hidden email]> wrote: > > Okay! I made some additional changes to my app/presenter/model so that I'm doing this: > > MyApp>>start > | model pres | > model := MyModel loadData. > pres := MyPresenterWithModel newApplication: self with: model. > pres openWithSpec. > > I changed my code in Presenter to reference the model - and it pops right up! When I close the GUI, GC and check for errant object instances... NONE!! I can open the UI over and over and my object instances stay at zero. > > So it feels like I'm on the right track. > > I still need a bunch of work/learning around the App/Presenter/Model interactions... so if anyone can point me to a good tutorial on this, including changing data and UI widgets - I'd greatly appreciate it. > > Thanks to all on this issue! > > Russ > > > On Wed, Apr 1, 2020 at 12:14 PM Russ Whaley <[hidden email]> wrote: > Tim, > Thanks for the reply! When I eliminated (with your become: solution), it did not a small chunk of the objects instances off. Another 'holder' of object instances accounted for another slug. But things still were not dropping like I hoped (this is without just setting everything to strings - but doing so systematically by object to perhaps see what was holding on to things). Finally I got things down to a few objects with instances in the hundreds and thousands - instead of 50,000 and 400,000 - and I finally set everything to String new. Interestingly enough, after garbage collection, when I do String instanceCount - I get 0 (zero) instances. > > All that - and thank you again - I closed all the system browsers, checked playground to delete bindings, I did a bunch of garbageCollection runs, did a System>Do Image Cleanup... then did a Save As. > > And... it is 8MB bigger than before I did all this cleanup. My image was 346MB and grew to 354MB AFTER the cleanup. Go figure. Maybe it will automagically GC sometime. > > I'm looking into what you and Sven mentioned - the pointersTo - I'm trying to figure out how to use it. > > A couple of basic questions... > • I thought tempVariables, inside a method were automatically destroyed when the method terminated? > • If I pass an object (either a parm or tempVariable) that resides in that message to another message, since I'm passing the actual object - I curious how it ever gets GC'd. > • All my ObjectTest classes create their own data to work with - I have the creation in centralized messages, but they return 'fresh' data for each test. Whatever the answer is in the first 2 bullet points, this could be a contributor to the volume. > I'm running some tests of the GUI after clearing out all the Object instances... and I'm seeing the growth as expected... it just doesn't go away when I close the GUI window. I'm not using SpApp correctly, nor am I using SpPresenter correctly (I'm forcing my model data into an SpPresenter instVar (via the SpApp start) so the presenter can use my data. This is all wrong but I'm struggling finding good examples in Spec2 of how to get these to work together. > > Thanks for the info - your suggestions have been very helpful! > Russ > > On Wed, Apr 1, 2020 at 2:43 AM Tim Mackinnon <[hidden email]> wrote: > Hi Russ - a quick look at your stats seems to indicate that something is holding on to your model, I didn’t understand the first object growing by 2, but the second increases by 1 each time, and it looks like some form of root object holding on to the others ? This of course means the GC can’t reclaim the chain of objects if something is holding on to that root globally. > > Assigning something in the playground will hold onto it (I believe) so you can continue to reuse and inspect it - so that’s expected, but this looks like something in your UI? > > You can break the chain manually (but it’s not a full solution), simply iterate over your root and become: String new (Save your image before doing it and see if it works to free up the cycles). Eg try something like this (haven’t tried this in Pharo myself, but worked in VA, so it should work here) > > MyRootModel allInstances do: [ :m | m become: String new ] > > If this works and reduces your memory usage, you now need to inspect one of those MyRootModel instances and see who is referencing it, and explain why. This is Sven’s pointerTo explanation. > > For roots, you sometimes can use a WeakDictionary so that when nothing is referencing them, they get gc’d if you are doing some caching or have some factory concept. > > It’s not uncommon to hit this when you get your system nearing completion , it’s the downside of not having to worry about memory referencing - ultimately you have to worry about it at the edges. > > It is possible there is a Pharo bug, as over time I see large Pharo images but I was just messing around and put it down to failed experiments. > > Hope this helps. > > Tim > >> On 31 Mar 2020, at 20:47, Russ Whaley <[hidden email]> wrote: >> >> >> Here is some additional data - attached - I checked the number of instances on select Classes before a UI load, during the UI load and after the UI load (and again). The class instances grow as expected, but do no release. I'm doing something wrong. I'm going to try a fresh image next, however, I expect the same thing to happen, just starting over at zero. >> >> >> >> On Tue, Mar 31, 2020 at 12:57 PM Russ Whaley <[hidden email]> wrote: >> Dario, >> Thanks for the reply! >> >> When I closed my image - after running that huge query and leaving it in an inspector... my image grew to 1.3GB, lol. >> >> When I closed the inspector, and all the class browsers, did the garbage collection, etc., it dropped back to 384MB. Still huge, and all my Class instances are still there. I'm getting ready to also drop my Playground - after I copy all the code snippets out and see if that has any impact. I'm also going to try a fresh image and see if there is the same growth pattern as here. >> >> Perhaps I'm doing something wrong with the way I'm creating and storing my object instances - which is generally Class new, fill out some attributes, then add it to a collection or Dictionary. >> >> All my Object tests create sample data (objects) - over and over as well. >> >> >> On Tue, Mar 31, 2020 at 12:20 PM dario.trussardi65 <[hidden email]> wrote: >> Ciao, >> >> >> > My image size keeps growing and growing. I've been scouring Google and mailing lists, trying to find a solution. >> >> i have the same problem with a Pharo 7 image. >> >> Maybe it has nothing to do. >> >> But after close all the class browser windows the image save return to a " valid " size. >> >> Dario >> >> >> -- >> Russ Whaley >> [hidden email] >> >> >> -- >> Russ Whaley >> [hidden email] >> <Class instances tests.pdf> > > > -- > Russ Whaley > [hidden email] > > > -- > Russ Whaley > [hidden email] |
Re: Ballooning image size! Approaching 400MB
|
Administrator
|
On Wed, Apr 1, 2020 at 11:17 AM Sven Van Caekenberghe <[hidden email]> wrote: Hi Russ, Russ, I didn't see anyone really answer these two points. It is important to understand that a variable, whether temporary or instance, is a holder of a reference to an object. It's not the object, per se. So, yes, when that method exits, the temporary variable on the stack frame is dropped, effectively eliminating the reference its temporary variable held. That formerly referenced object will be garbage collected when the automatic garbage collection finds it has no reachable references to it at all. "reachable" is the key to GC. If traversing the VM's object graph, the garbage collector cannot reach an object, then it is garbage, even if it has a million references (from other also unreachable objects). Things are slightly complicated by the notion of a weak reference. The garbage collector doesn't follow a weak reference, so if an object is only reachable via weak references, it is considered unreachable and can be collected. > • All my ObjectTest classes create their own data to work with - I have the creation in centralized messages, but they return 'fresh' data for each test. Whatever the answer is in the first 2 bullet points, this could be a contributor to the volume. |
Re: Ballooning image size! Approaching 400MB
|
|
In reply to this post by Russ Whaley
Hi Russ -
For the first 1, its not so much the “temp” ness of variables, but more about how the object they point to gets referenced. Normally in a method, you assign an object to a variable, you use it, nothing else references it, and it just gets GC’d. However if you call a setter: of another object, and pass in your temp variable as an argument, that other object now has a reference to it. Now if that other object is referenced in a class variable (or some other global), the gc will consider it referenced and won’t eliminate it. Basically every object reference gets traversed to see if its pinned in some way. Those objects that are not pinned (i.e. held onto anymore) get collected. There are numerous algorithms to do this efficiently, and you can read up on them (the basic one is mark and sweep). I recall that a global variable is anything that can be traced back to the Smalltalk system dictionary (which is the root of the world). There might be some other known roots (this is my vague memory on this. The above should answer your question #2 (if not, I’m sure we can find the relevant docs on this, its quite well explained somewhere). For #3, it shouldn’t be the volume of data created - its what is holding on to this data - do you cache it somehow (and end up referencing it in a global variable so it can’t be gc’d) - OR - if its just UI related (this might be a good test, if you create your model without a UI, and print something on the transcript, does memory still grow?). If its UI related - this could be some Spec handler that reaches back into the global system dictionary to process events (or something like this), and this is what is pinning your model. Or is there some phantom window that is tied to OS resources which stop it from properly terminating (but might not be visible). This is the kind of thing you need to look for. Spec2 is quite new, so it could be something there that is causing this. If you suspect its UI related, maybe you can create a really simple example with a simpler model subset that can recreate it. Hope this helps guide you on next steps. Tim
|
Re: Ballooning image size! Approaching 400MB
|
|
In reply to this post by Richard Sargent
Thank you I appreciate the explanation. On Wed, Apr 1, 2020 at 2:56 PM Richard Sargent <[hidden email]> wrote:
Russ Whaley [hidden email] |
«
Return to Pharo Smalltalk Users
|
1 view|%1 views
| Free forum by Nabble | Edit this page |