Serious garbage/storage leak issue with MCInfoProxy
Locked
12
12
Serious garbage/storage leak issue with MCInfoProxy
 
|
Hi All,
I've been experiencing image save slowdowns recently and finally my work image reached 1.%Gb and I thought I better take a look: Sisyphus.Cog$ ls -lh SpurWork64.* save/SpurWork64-* -rw-r--r--@ 1 eliot staff 28M Jan 18 12:47 SpurWork64.changes -rw-r--r--@ 1 eliot staff 1.6G Jan 18 12:48 SpurWork64.image -rw-r--r--@ 1 eliot staff 28M Jan 18 12:03 save/SpurWork64-2018-01-18.changes -rw-r--r--@ 1 eliot staff 1.5G Jan 18 12:03 save/SpurWork64-2018-01-18.image I ran a space analysis and found that Bitmap and ByteArray were the top two, so I looked for large Bitmaps. I found three that fit this criterion: Bitmap allInstances select: [:bm| bm size >= 1000000 and: [bm ~~ Display bits]] I inspected the three and did a chase pointers on one of them. As I did that suddenly a) the inspector on the Array became empty (still an array but zero elements) b) the progress bar for Downloading FlexibleVocabularies-who.NN appeared I interrupted this and did a very cursory stack examination. Some object had not understood isLiteral and from there what looked like an attempt to turn this stub into a real object caused FlexibleVocabularies-who.NN to start to download. I threw away the debugger, ran the GC and suddenly all my free space was back. So now on disc I have Sisyphus.Cog$ ls -lh SpurWork64.* save/SpurWork64-* -rw-r--r--@ 1 eliot staff 28M Jan 18 15:17 SpurWork64.changes -rw-r--r--@ 1 eliot staff 57M Jan 18 15:17 SpurWork64.image -rw-r--r--@ 1 eliot staff 28M Jan 18 12:03 save/SpurWork64-2018-01-18.changes -rw-r--r--@ 1 eliot staff 1.5G Jan 18 12:03 save/SpurWork64-2018-01-18.image What is going on here? There seems to be a very bad storage leak. Can we please discuss this? This doesn't seem like healthy behaviour at all :-) _,,,^..^,,,_ best, Eliot |
|
|
On Thu, Jan 18, 2018 at 3:24 PM, Eliot Miranda <[hidden email]> wrote:
My behavior was different. Also, my image is a bit smaller (300mb). I ran you check for large Bitmaps, and found 3 (all length 1924440). #chasePointers on each of them from the inspect chased them back to the inspecter. So I ran: (Bitmap allInstances select: [:bm| bm size >= 1000000 and: [bm ~~ Display bits]]) anyOne chasePointers This goes away for a long time - I got bored, interupted it, and inspected the large bitmaps again. This time, I only found 1. After saving the image, the image size did not drop (still 326mb). Image and VM is undoubttedly behind yours, but here it is: Image ----- C:\Squeak\Viz Landscape 6.0 64\hmm-64bit.1.image Squeak6.0alpha latest update: #17647 Image format 68021 (64 bit) Virtual Machine --------------- C:\Squeak\Viz Landscape 6.0 64\Squeak.exe Croquet Closure Stack VM [StackInterpreterPrimitives VMMaker.oscog-eem.2188] Win32 built on Apr 12 2017 09:46:07 GMT Compiler: 4.2.1 Compatible Clang 3.9.1 (tags/RELEASE_391/final) platform sources revision VM: 201704120850 https://github.com/OpenSmalltalk/opensmalltalk-vm.git $ Date: Wed Apr 12 10:50:48 2017 +0200 $ Plugins: 201704120850 https://github.com/OpenSmalltalk/opensmalltalk-vm.git $ StackInterpreter VMMaker.oscog-eem.2188 uuid: ff4ca601-cd05-4792-ab0d-dcdf19975239 Apr 12 2017 |
Re: Serious garbage/storage leak issue with MCInfoProxy
|
|
> On Thu, Jan 18, 2018 at 3:24 PM, Eliot Miranda <[hidden email]> wrote: > > I interrupted this and did a very cursory stack examination. Some object had not understood isLiteral and from there what looked like an attempt to turn this stub into a real object caused FlexibleVocabularies-who.NN to start to download. It seems extra-odd that the FlexibleVocabularies package seems to be empty these days. I see a tenuous connection from using the pointerchaser to #isLiteral (Object>>#shouldFollowOutboundPointers is sent from PointerFinder>>#follow:from:) but quite why that would lead to FlexibleVocabularies I have no idea. Maybe proxy things should over-ride #shouldFollowOutboundPointers ? tim -- tim Rowledge; [hidden email]; http://www.rowledge.org/tim Objects are closer than they appear. |
Re: Serious garbage/storage leak issue with MCInfoProxy
|
|
In reply to this post by Eliot Miranda-2
Hi Eliot,
It sounds like you selected 'flush cached versions and ancestry' from the Repository menu of the Monticello Working Copy browser. That's something that you would only want to do for a production deployment image where you aren't planning to do any more development in. It saves memory on ancestry while providing a dynamic (albeit, slow) means of loading it back on the rare case that one would need to do that in the production image ever again. The PointerFinder is not sensitive enough for proxies, so using it ends up agitating them into reification, forcing the dynamic reload of ancestry like you experienced. What you want is simply, "flush cached versions" and then you'll never have that issue. - Chris On Thu, Jan 18, 2018 at 5:24 PM, Eliot Miranda <[hidden email]> wrote: > Hi All, > > I've been experiencing image save slowdowns recently and finally my work > image reached 1.%Gb and I thought I better take a look: > > Sisyphus.Cog$ ls -lh SpurWork64.* save/SpurWork64-* > -rw-r--r--@ 1 eliot staff 28M Jan 18 12:47 SpurWork64.changes > -rw-r--r--@ 1 eliot staff 1.6G Jan 18 12:48 SpurWork64.image > -rw-r--r--@ 1 eliot staff 28M Jan 18 12:03 > save/SpurWork64-2018-01-18.changes > -rw-r--r--@ 1 eliot staff 1.5G Jan 18 12:03 > save/SpurWork64-2018-01-18.image > > I ran a space analysis and found that Bitmap and ByteArray were the top two, > so I looked for large Bitmaps. I found three that fit this criterion: > > > Bitmap allInstances select: [:bm| bm size >= 1000000 and: [bm ~~ Display > bits]] > > I inspected the three and did a chase pointers on one of them. As I did > that suddenly > a) the inspector on the Array became empty (still an array but zero > elements) > b) the progress bar for Downloading FlexibleVocabularies-who.NN appeared > > I interrupted this and did a very cursory stack examination. Some object had > not understood isLiteral and from there what looked like an attempt to turn > this stub into a real object caused FlexibleVocabularies-who.NN to start to > download. > > I threw away the debugger, ran the GC and suddenly all my free space was > back. So now on disc I have > > Sisyphus.Cog$ ls -lh SpurWork64.* save/SpurWork64-* > -rw-r--r--@ 1 eliot staff 28M Jan 18 15:17 SpurWork64.changes > -rw-r--r--@ 1 eliot staff 57M Jan 18 15:17 SpurWork64.image > -rw-r--r--@ 1 eliot staff 28M Jan 18 12:03 > save/SpurWork64-2018-01-18.changes > -rw-r--r--@ 1 eliot staff 1.5G Jan 18 12:03 > save/SpurWork64-2018-01-18.image > > What is going on here? There seems to be a very bad storage leak. Can we > please discuss this? This doesn't seem like healthy behaviour at all :-) > > _,,,^..^,,,_ > best, Eliot > > > |
Re: Serious garbage/storage leak issue with MCInfoProxy
|
|
Hi Chris,
On Thu, Jan 18, 2018 at 7:20 PM, Chris Muller <[hidden email]> wrote: Hi Eliot, Can you explain? How does this create the issue? Why have an item like "flush cached versions and ancestry" if its dangerous. The implication of :cached" is that the information can be rebuilt when required. The ancestry is out there in the .mcz files. I don't understand what's going on here. feels like a bad design decision.
_,,,^..^,,,_ best, Eliot |
Re: Serious garbage/storage leak issue with MCInfoProxy
|
|
It wasn't clear to me that your bloated image issue was related to
MCInfoProxy at all -- you said it showed ByteArray's and Bitmaps as the top, but the Proxy wasn't agitated until you ran the pointer-finder. As I mentioned, the intention of the design is to allow purging the MC ancestry to recover that memory for building a smaller, tighter production image, without losing the ability to access it, transparently, if needed. Because no one would want to accidentally save a version with the ancestry gone (corrupt), right? This design requires no extra thinking, just a little extra waiting. Conceptually, that feels like a classic cache, to me. If there's a glitch in how it interacts with MC that could be improved, we should fix it. I'm curious what else was on that stack.. I agree, though, it isn't needed on the UI menu, so we should remove it. It's something one would normally only call from a deployment/build script. Best, Chris On Thu, Jan 18, 2018 at 9:43 PM, Eliot Miranda <[hidden email]> wrote: > Hi Chris, > > On Thu, Jan 18, 2018 at 7:20 PM, Chris Muller <[hidden email]> wrote: >> >> Hi Eliot, >> >> It sounds like you selected 'flush cached versions and ancestry' from >> the Repository menu of the Monticello Working Copy browser. >> >> That's something that you would only want to do for a production >> deployment image where you aren't planning to do any more development >> in. It saves memory on ancestry while providing a dynamic (albeit, >> slow) means of loading it back on the rare case that one would need to >> do that in the production image ever again. The PointerFinder is not >> sensitive enough for proxies, so using it ends up agitating them into >> reification, forcing the dynamic reload of ancestry like you >> experienced. >> >> What you want is simply, "flush cached versions" and then you'll never >> have that issue. > > > Can you explain? How does this create the issue? Why have an item like > "flush cached versions and ancestry" if its dangerous. The implication of > :cached" is that the information can be rebuilt when required. The ancestry > is out there in the .mcz files. I don't understand what's going on here. > feels like a bad design decision. > >> >> >> - Chris >> >> On Thu, Jan 18, 2018 at 5:24 PM, Eliot Miranda <[hidden email]> >> wrote: >> > Hi All, >> > >> > I've been experiencing image save slowdowns recently and finally my >> > work >> > image reached 1.%Gb and I thought I better take a look: >> > >> > Sisyphus.Cog$ ls -lh SpurWork64.* save/SpurWork64-* >> > -rw-r--r--@ 1 eliot staff 28M Jan 18 12:47 SpurWork64.changes >> > -rw-r--r--@ 1 eliot staff 1.6G Jan 18 12:48 SpurWork64.image >> > -rw-r--r--@ 1 eliot staff 28M Jan 18 12:03 >> > save/SpurWork64-2018-01-18.changes >> > -rw-r--r--@ 1 eliot staff 1.5G Jan 18 12:03 >> > save/SpurWork64-2018-01-18.image >> > >> > I ran a space analysis and found that Bitmap and ByteArray were the top >> > two, >> > so I looked for large Bitmaps. I found three that fit this criterion: >> > >> > >> > Bitmap allInstances select: [:bm| bm size >= 1000000 and: [bm ~~ >> > Display >> > bits]] >> > >> > I inspected the three and did a chase pointers on one of them. As I did >> > that suddenly >> > a) the inspector on the Array became empty (still an array but zero >> > elements) >> > b) the progress bar for Downloading FlexibleVocabularies-who.NN appeared >> > >> > I interrupted this and did a very cursory stack examination. Some object >> > had >> > not understood isLiteral and from there what looked like an attempt to >> > turn >> > this stub into a real object caused FlexibleVocabularies-who.NN to start >> > to >> > download. >> > >> > I threw away the debugger, ran the GC and suddenly all my free space was >> > back. So now on disc I have >> > >> > Sisyphus.Cog$ ls -lh SpurWork64.* save/SpurWork64-* >> > -rw-r--r--@ 1 eliot staff 28M Jan 18 15:17 SpurWork64.changes >> > -rw-r--r--@ 1 eliot staff 57M Jan 18 15:17 SpurWork64.image >> > -rw-r--r--@ 1 eliot staff 28M Jan 18 12:03 >> > save/SpurWork64-2018-01-18.changes >> > -rw-r--r--@ 1 eliot staff 1.5G Jan 18 12:03 >> > save/SpurWork64-2018-01-18.image >> > >> > What is going on here? There seems to be a very bad storage leak. Can >> > we >> > please discuss this? This doesn't seem like healthy behaviour at all >> > :-) >> > >> > _,,,^..^,,,_ >> > best, Eliot >> > >> > >> > >> > > > > -- > _,,,^..^,,,_ > best, Eliot |
Re: Serious garbage/storage leak issue with MCInfoProxy
|
|
It wasn't clear to me that your bloated image issue was related to I've voiced my criticism of that design before, so I'll just point out that it's exactly this unpredictable nature of proxy interactions that makes it fragile. I agree, though, it isn't needed on the UI menu, so we should remove +1 If even Master Eliot is confused, we really should not expose it with one tempting click ;) - Bert -
|
Re: Serious garbage/storage leak issue with MCInfoProxy
|
+1 :-) Best, Marcel
|
Re: Serious garbage/storage leak issue with MCInfoProxy
|
|
In reply to this post by Bert Freudenberg
To what unpredictability and fragility do you refer? If you're
referring to the override of DNU, then you appear to be singling out this one use of it without proposing a better alternative or complaining about other uses of it as "unpredictable". Why? Given your own record advocating for the most insidious and dangerous types of bugs that silently corrupt data (notably, the deeply flawed ideas about the Auto-Timezone). I suppose you would prefer something that simply nil's out the ancestry so that when someone saves it, they're saving a corrupt package? That's would be a lot worse than an "unpredictable" delay. On Fri, Jan 19, 2018 at 5:50 AM, Bert Freudenberg <[hidden email]> wrote: > On 19 January 2018 at 05:30, Chris Muller <[hidden email]> wrote: >> >> It wasn't clear to me that your bloated image issue was related to >> MCInfoProxy at all -- you said it showed ByteArray's and Bitmaps as >> the top, but the Proxy wasn't agitated until you ran the >> pointer-finder. >> >> As I mentioned, the intention of the design is to allow purging the MC >> ancestry to recover that memory for building a smaller, tighter >> production image, without losing the ability to access it, >> transparently, if needed. Because no one would want to accidentally >> save a version with the ancestry gone (corrupt), right? This design >> requires >> no extra thinking, just a little extra waiting. Conceptually, that >> feels like a classic cache, to me. If there's a glitch in how it >> interacts with MC that could be improved, we should fix it. I'm >> curious what else was on that stack.. > > > I've voiced my criticism of that design before, so I'll just point out that > it's exactly this unpredictable nature of proxy interactions that makes it > fragile. > >> I agree, though, it isn't needed on the UI menu, so we should remove >> it. It's something one would normally only call from a >> deployment/build script. > > > +1 > > If even Master Eliot is confused, we really should not expose it with one > tempting click ;) > > - Bert - > > >> >> >> Best, >> Chris >> >> On Thu, Jan 18, 2018 at 9:43 PM, Eliot Miranda <[hidden email]> >> wrote: >> > Hi Chris, >> > >> > On Thu, Jan 18, 2018 at 7:20 PM, Chris Muller <[hidden email]> >> > wrote: >> >> >> >> Hi Eliot, >> >> >> >> It sounds like you selected 'flush cached versions and ancestry' from >> >> the Repository menu of the Monticello Working Copy browser. >> >> >> >> That's something that you would only want to do for a production >> >> deployment image where you aren't planning to do any more development >> >> in. It saves memory on ancestry while providing a dynamic (albeit, >> >> slow) means of loading it back on the rare case that one would need to >> >> do that in the production image ever again. The PointerFinder is not >> >> sensitive enough for proxies, so using it ends up agitating them into >> >> reification, forcing the dynamic reload of ancestry like you >> >> experienced. >> >> >> >> What you want is simply, "flush cached versions" and then you'll never >> >> have that issue. >> > >> > >> > Can you explain? How does this create the issue? Why have an item like >> > "flush cached versions and ancestry" if its dangerous. The implication >> > of >> > :cached" is that the information can be rebuilt when required. The >> > ancestry >> > is out there in the .mcz files. I don't understand what's going on >> > here. >> > feels like a bad design decision. >> > >> >> >> >> >> >> - Chris >> >> >> >> On Thu, Jan 18, 2018 at 5:24 PM, Eliot Miranda >> >> <[hidden email]> >> >> wrote: >> >> > Hi All, >> >> > >> >> > I've been experiencing image save slowdowns recently and finally >> >> > my >> >> > work >> >> > image reached 1.%Gb and I thought I better take a look: >> >> > >> >> > Sisyphus.Cog$ ls -lh SpurWork64.* save/SpurWork64-* >> >> > -rw-r--r--@ 1 eliot staff 28M Jan 18 12:47 SpurWork64.changes >> >> > -rw-r--r--@ 1 eliot staff 1.6G Jan 18 12:48 SpurWork64.image >> >> > -rw-r--r--@ 1 eliot staff 28M Jan 18 12:03 >> >> > save/SpurWork64-2018-01-18.changes >> >> > -rw-r--r--@ 1 eliot staff 1.5G Jan 18 12:03 >> >> > save/SpurWork64-2018-01-18.image >> >> > >> >> > I ran a space analysis and found that Bitmap and ByteArray were the >> >> > top >> >> > two, >> >> > so I looked for large Bitmaps. I found three that fit this >> >> > criterion: >> >> > >> >> > >> >> > Bitmap allInstances select: [:bm| bm size >= 1000000 and: [bm ~~ >> >> > Display >> >> > bits]] >> >> > >> >> > I inspected the three and did a chase pointers on one of them. As I >> >> > did >> >> > that suddenly >> >> > a) the inspector on the Array became empty (still an array but zero >> >> > elements) >> >> > b) the progress bar for Downloading FlexibleVocabularies-who.NN >> >> > appeared >> >> > >> >> > I interrupted this and did a very cursory stack examination. Some >> >> > object >> >> > had >> >> > not understood isLiteral and from there what looked like an attempt >> >> > to >> >> > turn >> >> > this stub into a real object caused FlexibleVocabularies-who.NN to >> >> > start >> >> > to >> >> > download. >> >> > >> >> > I threw away the debugger, ran the GC and suddenly all my free space >> >> > was >> >> > back. So now on disc I have >> >> > >> >> > Sisyphus.Cog$ ls -lh SpurWork64.* save/SpurWork64-* >> >> > -rw-r--r--@ 1 eliot staff 28M Jan 18 15:17 SpurWork64.changes >> >> > -rw-r--r--@ 1 eliot staff 57M Jan 18 15:17 SpurWork64.image >> >> > -rw-r--r--@ 1 eliot staff 28M Jan 18 12:03 >> >> > save/SpurWork64-2018-01-18.changes >> >> > -rw-r--r--@ 1 eliot staff 1.5G Jan 18 12:03 >> >> > save/SpurWork64-2018-01-18.image >> >> > >> >> > What is going on here? There seems to be a very bad storage leak. >> >> > Can >> >> > we >> >> > please discuss this? This doesn't seem like healthy behaviour at all >> >> > :-) >> >> > >> >> > _,,,^..^,,,_ >> >> > best, Eliot >> >> > >> >> > >> >> > >> >> >> > >> > >> > >> > -- >> > _,,,^..^,,,_ >> > best, Eliot >> > > > > |
Re: Serious garbage/storage leak issue with MCInfoProxy
|
|
The problem I have with proxies *as a user* is that when you’re doing something where they help, they're great and when you do something where they surprise you and trigger some time-consuming thing they’re terrible. Finding a middle ground may be difficult.
In a case where something triggers a load of packages - which is almost always going to be pretty time-consuming, even when you have gigabit internet and the package server is in a really good mood - it would be nice to have some opportunity to stop it. In fact I think it would be nice to have a reasonably clean ‘stop this right now!’ button on the progress morph for almost any kind of long-running activity. tim -- tim Rowledge; [hidden email]; http://www.rowledge.org/tim Strange OpCodes: SLTMDL: Shift Left, Test Mask and Dim the Lights |
Re: Serious garbage/storage leak issue with MCInfoProxy
|
|
I'm not beholden to the "proxy" implementation, just the
design-properties that it provides: 1) achieves the goal of purging / recovering the memory consumed by the ancestry objects while, 2) doing so safely and transparently, with never any intention of mutilating or dis-membering the ancestry -- because the stub is as good as the original when called upon. Those are the _design_ properties anyway, if there is some other implementation which could achieve these properties, I would support it. As for the time-consuming reification, I like Tim's idea of a cancel button on the Progress bar (note that Eliot DID simply use Command+dot) rather than a confirmation prompt because of the headless server situation -- a delay would be painful enough to notice, but not enough to bring the server down like a confirmation prompt sitting there... I'll at least get it removed from the menu for now. On Fri, Jan 19, 2018 at 12:47 PM, tim Rowledge <[hidden email]> wrote: > The problem I have with proxies *as a user* is that when you’re doing something where they help, they're great and when you do something where they surprise you and trigger some time-consuming thing they’re terrible. Finding a middle ground may be difficult. > > In a case where something triggers a load of packages - which is almost always going to be pretty time-consuming, even when you have gigabit internet and the package server is in a really good mood - it would be nice to have some opportunity to stop it. In fact I think it would be nice to have a reasonably clean ‘stop this right now!’ button on the progress morph for almost any kind of long-running activity. > > tim > -- > tim Rowledge; [hidden email]; http://www.rowledge.org/tim > Strange OpCodes: SLTMDL: Shift Left, Test Mask and Dim the Lights > > |
Re: Serious garbage/storage leak issue with MCInfoProxy
|
|
In reply to this post by Chris Muller-3
Hi Chris,
On Fri, Jan 19, 2018 at 10:30 AM, Chris Muller <[hidden email]> wrote: To what unpredictability and fragility do you refer? If you're I have no objections to the proxies. It was the presence of something for deployment that is on a user menu that freaked me out. In fact, if I understand it correctly flush ancestry replaces the ancestry with proxies that will auto-load the ancestry on first access? If that's so I actually have no objection. Dat is not being used. What I had misunderstood from the earlier messages was that flush cached versions and ancestry destroyed ancestry. As long as that's not true I'm happy. I'm still alarmed that my now 54Mb image ballooned to 1.5Gb until it appears I somehow kicked off a cleanup by touching the ancestry proxies. But that's a different issue.
I've upset you. I'm sorry. The latter is what I had misunderstood the situation to be. Forgive me; when I'm debugging the VM I find myself at the limit of my capacities and when, in the course of bug fixing, I have to interact with some unfamiliar piece of the system I get frustrated.
I stand corrected.
_,,,^..^,,,_ best, Eliot |
Re: Serious garbage/storage leak issue with MCInfoProxy
|
|
On Fri, Jan 19, 2018 at 3:48 PM, Eliot Miranda <[hidden email]> wrote:
> Hi Chris, > > On Fri, Jan 19, 2018 at 10:30 AM, Chris Muller <[hidden email]> wrote: >> >> To what unpredictability and fragility do you refer? If you're >> referring to the override of DNU, then you appear to be singling out >> this one use of it without proposing a better alternative or >> complaining about other uses of it as "unpredictable". Why? > > > I have no objections to the proxies. It was the presence of something for > deployment that is on a user menu that freaked me out. I just committed it's removal from the menu. I agree that it should not be on there. > In fact, if I > understand it correctly flush ancestry replaces the ancestry with proxies > that will auto-load the ancestry on first access? Yes. > If that's so I actually > have no objection. Dat is not being used. What I had misunderstood from > the earlier messages was that flush cached versions and ancestry destroyed > ancestry. As long as that's not true I'm happy. Good. If you're happy, I'm happy. :) > I'm still alarmed that my now 54Mb image ballooned to 1.5Gb until it appears > I somehow kicked off a cleanup by touching the ancestry proxies. But that's > a different issue. That does sound strange. The next time I use this, I'll keep my eye on that. > I've upset you. I'm sorry. The latter is what I had misunderstood the > situation to be. Forgive me; when I'm debugging the VM I find myself at the > limit of my capacities and when, in the course of bug fixing, I have to > interact with some unfamiliar piece of the system I get frustrated. No no, I was not upset, above I was being cheeky to Bert as a way to say I prefer to discuss solutions, not opinions. I'm sorry for coming across like a brillo pad. I know exactly where you're coming from about running into something unexpected when one is deep in a zone. Getting yanked out of that mental "deep stack" that took a lot to get there is.. very frustrating. Sorry, Eliot. Best, Chris |
Re: Serious garbage/storage leak issue with MCInfoProxy
|
|
In reply to this post by Eliot Miranda-2
Hi Marcel, I've finally worked out that the space overhead here comes from the dropShadow cache system in Morphs (in otherProperties in MorphExtensions). Would it be possible to arrange to flush all drop shadows in the current project when: - exiting a project - snapshotting ? On Thu, Jan 18, 2018 at 3:24 PM, Eliot Miranda <[hidden email]> wrote:
_,,,^..^,,,_ best, Eliot |
Re: Serious garbage/storage leak issue with MCInfoProxy
|
|
Hi Eliot, sure. Removing the potential drop shadow of all kinds of morphs takes time: Morph allSubInstancesDo: [:ea | ea removeProperty: #dropShadow]. About 3 seconds here in a quite clean image. SystemWindow allSubInstancesDo: [:ea | ea removeProperty: #dropShadow]. Works at 100 milliseconds. What has the biggest effect in your image? Best, Marcel
|
Re: Serious garbage/storage leak issue with MCInfoProxy
|
|
Probably good enough might be World allMorphsDo: [:ea | ea removeProperty: #dropShadow] takes 0 ms for me On 2/20/18 2:46 AM, Marcel Taeumel
wrote:
|
Re: Serious garbage/storage leak issue with MCInfoProxy
|
|
We have to consider all open projects: Project allMorphicProjects do: [:project | project world allMorphsDo: [:morph | morph removeProperty: #dropShadow]]. Yet, there can be hidden morphs. If we want to ignore those, we can stick with only cleaning up top-level morphs, which are usually the system windows: Project allMorphicProjects do: [:project | project world submorphsDo: [:morph | morph removeProperty: #dropShadow]]. Best, Marcel
|
Re: Serious garbage/storage leak issue with MCInfoProxy
|
|
Hi Marcel,
Remember that the drop shadows are all destroyed (or at least appear to be) when one does a save and quit. Bloating happens when, as I do, one repeatedly saves an image and one does not quit. Starting an image results in a system that only has drop shadows in the active world. So there is no need to establish the invariant by e.g. adding a package script, but if one wanted one could put the slow trawl through all morphs in a package script. But you can speed up the trawl through all morphs by using MorphExtension allInstancesDo: [:morphExtension| ... since drop shadows are only held in MorphExtensions, and that should speed up the initial scan to void drop shadows to establish the invariant that there are no drop shadows in other than the current world. Cheers!
|
Re: Serious garbage/storage leak issue with MCInfoProxy
|
|
In reply to this post by marcel.taeumel
To handle all of the morphic projects, it would be better to do the
cleanup when exiting the projects: MorphicProject>>finalExitActions: enteringProject world allMorphsDo: [:ea | ea removeProperty: #dropShadow]. ... That will leave only the current project to worry about when saving the image. Of course, you do not want to to cleam up the morphs if the current world is a ControlManager, so it might look something like this: | world | (world := Project current world) isMorph ifTrue: [world allMorphsDo: [:ea | ea removeProperty: #dropShadow]] Dave On Tue, Feb 20, 2018 at 11:26:19AM +0100, Marcel Taeumel wrote: > We have to consider all open projects: > > Project allMorphicProjects do: [:project | > ?? ??project world allMorphsDo: [:morph | > ?? ?? ?? morph removeProperty: #dropShadow]]. > > Yet, there can be hidden morphs. If we want to ignore those, we can stick with only cleaning up top-level morphs, which are usually the system windows: > > Project allMorphicProjects do: [:project | > ?? ??project world submorphsDo: [:morph | > ?? ?? ?? morph removeProperty: #dropShadow]]. > > Best, > Marcel > Am 20.02.2018 10:57:08 schrieb Bob Arning <[hidden email]>: > Probably good enough might be > World allMorphsDo: [:ea | ea removeProperty: #dropShadow] > takes 0 ms for me > > > On 2/20/18 2:46 AM, Marcel Taeumel wrote: > > Hi Eliot, > > sure. Removing the potential drop shadow of all kinds of morphs takes time: > > Morph allSubInstancesDo: [:ea |??ea removeProperty: #dropShadow]. > > About 3 seconds here in a quite clean image. > > SystemWindow allSubInstancesDo: [:ea |??ea removeProperty: #dropShadow]. > > Works at 100 milliseconds. > > What has the biggest effect in your image? > > Best, > Marcel > Am 20.02.2018 02:51:06 schrieb Eliot Miranda <[hidden email]> [mailto:[hidden email]]: > Hi Marcel, > > ?? ?? I've finally worked out that the space overhead here comes from the dropShadow cache system in Morphs (in otherProperties in MorphExtensions).?? Would it be possible to arrange to flush all drop shadows in the current project when: > - exiting a project > - snapshotting > ? > > On Thu, Jan 18, 2018 at 3:24 PM, Eliot Miranda <[hidden email] [mailto:[hidden email]]> wrote: > > Hi All, > > ?? ?? I've been experiencing image save slowdowns recently and finally my work image reached 1.%Gb and I thought I better take a look: > > Sisyphus.Cog$ ls -lh SpurWork64.* save/SpurWork64-* > -rw-r--r--@ 1 eliot ??staff ?? ??28M Jan 18 12:47 SpurWork64.changes > -rw-r--r--@ 1 eliot ??staff ?? 1.6G Jan 18 12:48 SpurWork64.image > -rw-r--r--@ 1 eliot ??staff ?? ??28M Jan 18 12:03 save/SpurWork64-2018-01-18.changes > -rw-r--r--@ 1 eliot ??staff ?? 1.5G Jan 18 12:03 save/SpurWork64-2018-01-18.image > > I ran a space analysis and found that Bitmap and ByteArray were the top two, so I looked for large Bitmaps.?? I found three that fit this criterion: > > > ?? ?? Bitmap allInstances select: [:bm| bm size >= 1000000 and: [bm ~~ Display bits]] > > I inspected the three and did a chase pointers on one of them.?? As I did that suddenly > a) the inspector on the Array became empty (still an array but zero elements) > b) the progress bar for Downloading FlexibleVocabularies-who.NN appeared > > I interrupted this and did a very cursory stack examination. Some object had not understood isLiteral and from there what looked like an attempt to turn this stub into a real object caused FlexibleVocabularies-who.NN to start to download. > > I threw away the debugger, ran the GC and suddenly all my free space was back.?? So now on disc I have > > Sisyphus.Cog$ ls -lh SpurWork64.* save/SpurWork64-* > -rw-r--r--@ 1 eliot ??staff ?? ??28M Jan 18 15:17 SpurWork64.changes > -rw-r--r--@ 1 eliot ??staff ?? ??57M Jan 18 15:17 SpurWork64.image > -rw-r--r--@ 1 eliot ??staff ?? ??28M Jan 18 12:03 save/SpurWork64-2018-01-18.changes > -rw-r--r--@ 1 eliot ??staff ?? 1.5G Jan 18 12:03 save/SpurWork64-2018-01-18.image > > What is going on here??? There seems to be a very bad storage leak.?? Can we please discuss this??? This doesn't seem like healthy behaviour at all :-) > > _,,,^..^,,,_ > > best,??Eliot > > > > -- > > _,,,^..^,,,_ > > best,??Eliot > > > |
Re: Serious garbage/storage leak issue with MCInfoProxy
|
|
This might also justify adding one new method to Project:
Project>>cleanUpForExit Default implementation would be do nothing, and for MorphicProject it would clean up the drop morphs. This would be called from #finalExitActions: for ongoing cleanup, and when saving the image you would do this: Project current cleanupForExit Dave On Tue, Feb 20, 2018 at 07:50:57AM -0500, David T. Lewis wrote: > To handle all of the morphic projects, it would be better to do the > cleanup when exiting the projects: > > MorphicProject>>finalExitActions: enteringProject > world allMorphsDo: [:ea | ea removeProperty: #dropShadow]. > ... > > That will leave only the current project to worry about when saving > the image. Of course, you do not want to to cleam up the morphs if > the current world is a ControlManager, so it might look something > like this: > > | world | > (world := Project current world) isMorph > ifTrue: [world allMorphsDo: [:ea | ea removeProperty: #dropShadow]] > > Dave > > On Tue, Feb 20, 2018 at 11:26:19AM +0100, Marcel Taeumel wrote: > > We have to consider all open projects: > > > > Project allMorphicProjects do: [:project | > > ?? ??project world allMorphsDo: [:morph | > > ?? ?? ?? morph removeProperty: #dropShadow]]. > > > > Yet, there can be hidden morphs. If we want to ignore those, we can stick with only cleaning up top-level morphs, which are usually the system windows: > > > > Project allMorphicProjects do: [:project | > > ?? ??project world submorphsDo: [:morph | > > ?? ?? ?? morph removeProperty: #dropShadow]]. > > > > Best, > > Marcel > > Am 20.02.2018 10:57:08 schrieb Bob Arning <[hidden email]>: > > Probably good enough might be > > World allMorphsDo: [:ea | ea removeProperty: #dropShadow] > > takes 0 ms for me > > > > > > On 2/20/18 2:46 AM, Marcel Taeumel wrote: > > > > Hi Eliot, > > > > sure. Removing the potential drop shadow of all kinds of morphs takes time: > > > > Morph allSubInstancesDo: [:ea |??ea removeProperty: #dropShadow]. > > > > About 3 seconds here in a quite clean image. > > > > SystemWindow allSubInstancesDo: [:ea |??ea removeProperty: #dropShadow]. > > > > Works at 100 milliseconds. > > > > What has the biggest effect in your image? > > > > Best, > > Marcel > > Am 20.02.2018 02:51:06 schrieb Eliot Miranda <[hidden email]> [mailto:[hidden email]]: > > Hi Marcel, > > > > ?? ?? I've finally worked out that the space overhead here comes from the dropShadow cache system in Morphs (in otherProperties in MorphExtensions).?? Would it be possible to arrange to flush all drop shadows in the current project when: > > - exiting a project > > - snapshotting > > ? > > > > On Thu, Jan 18, 2018 at 3:24 PM, Eliot Miranda <[hidden email] [mailto:[hidden email]]> wrote: > > > > Hi All, > > > > ?? ?? I've been experiencing image save slowdowns recently and finally my work image reached 1.%Gb and I thought I better take a look: > > > > Sisyphus.Cog$ ls -lh SpurWork64.* save/SpurWork64-* > > -rw-r--r--@ 1 eliot ??staff ?? ??28M Jan 18 12:47 SpurWork64.changes > > -rw-r--r--@ 1 eliot ??staff ?? 1.6G Jan 18 12:48 SpurWork64.image > > -rw-r--r--@ 1 eliot ??staff ?? ??28M Jan 18 12:03 save/SpurWork64-2018-01-18.changes > > -rw-r--r--@ 1 eliot ??staff ?? 1.5G Jan 18 12:03 save/SpurWork64-2018-01-18.image > > > > I ran a space analysis and found that Bitmap and ByteArray were the top two, so I looked for large Bitmaps.?? I found three that fit this criterion: > > > > > > ?? ?? Bitmap allInstances select: [:bm| bm size >= 1000000 and: [bm ~~ Display bits]] > > > > I inspected the three and did a chase pointers on one of them.?? As I did that suddenly > > a) the inspector on the Array became empty (still an array but zero elements) > > b) the progress bar for Downloading FlexibleVocabularies-who.NN appeared > > > > I interrupted this and did a very cursory stack examination. Some object had not understood isLiteral and from there what looked like an attempt to turn this stub into a real object caused FlexibleVocabularies-who.NN to start to download. > > > > I threw away the debugger, ran the GC and suddenly all my free space was back.?? So now on disc I have > > > > Sisyphus.Cog$ ls -lh SpurWork64.* save/SpurWork64-* > > -rw-r--r--@ 1 eliot ??staff ?? ??28M Jan 18 15:17 SpurWork64.changes > > -rw-r--r--@ 1 eliot ??staff ?? ??57M Jan 18 15:17 SpurWork64.image > > -rw-r--r--@ 1 eliot ??staff ?? ??28M Jan 18 12:03 save/SpurWork64-2018-01-18.changes > > -rw-r--r--@ 1 eliot ??staff ?? 1.5G Jan 18 12:03 save/SpurWork64-2018-01-18.image > > > > What is going on here??? There seems to be a very bad storage leak.?? Can we please discuss this??? This doesn't seem like healthy behaviour at all :-) > > > > _,,,^..^,,,_ > > > > best,??Eliot > > > > > > > > -- > > > > _,,,^..^,,,_ > > > > best,??Eliot > > > > > > > > > |
«
Return to Squeak - Dev
|
1 view|%1 views
| Free forum by Nabble | Edit this page |