Serious garbage/storage leak issue with MCInfoProxy

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
28 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Re: Serious garbage/storage leak issue with MCInfoProxy

marcel.taeumel
Hmm... I do not see value in cleaning up projects (to free memory) when the user just switches between them back and forth. Especially in the light of worlds-in-worlds or background projects, this seems like an unreasonable path to follow. We should not make too much assumptions about application code or user needs at this point. There could always be a "Oh, please render that background project into this file. Oh, why is it so slow at the moment..."

Cleaning up resources for save-and-quit, however, seems reasonable because it is at the environment's boundary and not some arbitrary in-image boundary.

Best,
Marcel

Am 20.02.2018 13:58:38 schrieb David T. Lewis <[hidden email]>:

This might also justify adding one new method to Project:

Project>>cleanUpForExit

Default implementation would be do nothing, and for MorphicProject it
would clean up the drop morphs. This would be called from #finalExitActions:
for ongoing cleanup, and when saving the image you would do this:

Project current cleanupForExit

Dave


On Tue, Feb 20, 2018 at 07:50:57AM -0500, David T. Lewis wrote:

> To handle all of the morphic projects, it would be better to do the
> cleanup when exiting the projects:
>
> MorphicProject>>finalExitActions: enteringProject
> world allMorphsDo: [:ea | ea removeProperty: #dropShadow].
> ...
>
> That will leave only the current project to worry about when saving
> the image. Of course, you do not want to to cleam up the morphs if
> the current world is a ControlManager, so it might look something
> like this:
>
> | world |
> (world := Project current world) isMorph
> ifTrue: [world allMorphsDo: [:ea | ea removeProperty: #dropShadow]]
>
> Dave
>
> On Tue, Feb 20, 2018 at 11:26:19AM +0100, Marcel Taeumel wrote:
> > We have to consider all open projects:
> >
> > Project allMorphicProjects do: [:project |
> > ?? ??project world allMorphsDo: [:morph |
> > ?? ?? ?? morph removeProperty: #dropShadow]].
> >
> > Yet, there can be hidden morphs. If we want to ignore those, we can stick with only cleaning up top-level morphs, which are usually the system windows:
> >
> > Project allMorphicProjects do: [:project |
> > ?? ??project world submorphsDo: [:morph |
> > ?? ?? ?? morph removeProperty: #dropShadow]].
> >
> > Best,
> > Marcel
> > Am 20.02.2018 10:57:08 schrieb Bob Arning :
> > Probably good enough might be
> > World allMorphsDo: [:ea | ea removeProperty: #dropShadow]
> > takes 0 ms for me
> >
> >
> > On 2/20/18 2:46 AM, Marcel Taeumel wrote:
> >
> > Hi Eliot,
> >
> > sure. Removing the potential drop shadow of all kinds of morphs takes time:
> >
> > Morph allSubInstancesDo: [:ea |??ea removeProperty: #dropShadow].
> >
> > About 3 seconds here in a quite clean image.
> >
> > SystemWindow allSubInstancesDo: [:ea |??ea removeProperty: #dropShadow].
> >
> > Works at 100 milliseconds.
> >
> > What has the biggest effect in your image?
> >
> > Best,
> > Marcel
> > Am 20.02.2018 02:51:06 schrieb Eliot Miranda [mailto:[hidden email]]:
> > Hi Marcel,
> >
> > ?? ?? I've finally worked out that the space overhead here comes from the dropShadow cache system in Morphs (in otherProperties in MorphExtensions).?? Would it be possible to arrange to flush all drop shadows in the current project when:
> > - exiting a project
> > - snapshotting
> > ?
> >
> > On Thu, Jan 18, 2018 at 3:24 PM, Eliot Miranda wrote:
> >
> > Hi All,
> >
> > ?? ?? I've been experiencing image save slowdowns recently and finally my work image reached 1.%Gb and I thought I better take a look:
> >
> > Sisyphus.Cog$ ls -lh SpurWork64.* save/SpurWork64-*
> > -rw-r--r--@ 1 eliot ??staff ?? ??28M Jan 18 12:47 SpurWork64.changes
> > -rw-r--r--@ 1 eliot ??staff ?? 1.6G Jan 18 12:48 SpurWork64.image
> > -rw-r--r--@ 1 eliot ??staff ?? ??28M Jan 18 12:03 save/SpurWork64-2018-01-18.changes
> > -rw-r--r--@ 1 eliot ??staff ?? 1.5G Jan 18 12:03 save/SpurWork64-2018-01-18.image
> >
> > I ran a space analysis and found that Bitmap and ByteArray were the top two, so I looked for large Bitmaps.?? I found three that fit this criterion:
> >
> >
> > ?? ?? Bitmap allInstances select: [:bm| bm size >= 1000000 and: [bm ~~ Display bits]]
> >
> > I inspected the three and did a chase pointers on one of them.?? As I did that suddenly
> > a) the inspector on the Array became empty (still an array but zero elements)
> > b) the progress bar for Downloading FlexibleVocabularies-who.NN appeared
> >
> > I interrupted this and did a very cursory stack examination. Some object had not understood isLiteral and from there what looked like an attempt to turn this stub into a real object caused FlexibleVocabularies-who.NN to start to download.
> >
> > I threw away the debugger, ran the GC and suddenly all my free space was back.?? So now on disc I have
> >
> > Sisyphus.Cog$ ls -lh SpurWork64.* save/SpurWork64-*
> > -rw-r--r--@ 1 eliot ??staff ?? ??28M Jan 18 15:17 SpurWork64.changes
> > -rw-r--r--@ 1 eliot ??staff ?? ??57M Jan 18 15:17 SpurWork64.image
> > -rw-r--r--@ 1 eliot ??staff ?? ??28M Jan 18 12:03 save/SpurWork64-2018-01-18.changes
> > -rw-r--r--@ 1 eliot ??staff ?? 1.5G Jan 18 12:03 save/SpurWork64-2018-01-18.image
> >
> > What is going on here??? There seems to be a very bad storage leak.?? Can we please discuss this??? This doesn't seem like healthy behaviour at all :-)
> >
> > _,,,^..^,,,_
> >
> > best,??Eliot
> >
> >
> >
> > --
> >
> > _,,,^..^,,,_
> >
> > best,??Eliot
> >
> >
>
> >
>
>



Reply | Threaded
Open this post in threaded view
|

Re: Serious garbage/storage leak issue with MCInfoProxy

Eliot Miranda-2
Hi Marcel,


On Feb 20, 2018, at 5:00 AM, Marcel Taeumel <[hidden email]> wrote:

Hmm... I do not see value in cleaning up projects (to free memory) when the user just switches between them back and forth. Especially in the light of worlds-in-worlds or background projects, this seems like an unreasonable path to follow. We should not make too much assumptions about application code or user needs at this point. There could always be a "Oh, please render that background project into this file. Oh, why is it so slow at the moment..."

Well one might want to make the drop shadow caches responsive to some overall memory budget because, as I've observed, the overhead of the drop shadows can be many megabytes.  I saw an overhead that approached 1Gb!!

Cleaning up resources for save-and-quit, however, seems reasonable because it is at the environment's boundary and not some arbitrary in-image boundary.

Best,
Marcel

Am 20.02.2018 13:58:38 schrieb David T. Lewis <[hidden email]>:

This might also justify adding one new method to Project:

Project>>cleanUpForExit

Default implementation would be do nothing, and for MorphicProject it
would clean up the drop morphs. This would be called from #finalExitActions:
for ongoing cleanup, and when saving the image you would do this:

Project current cleanupForExit

Dave


On Tue, Feb 20, 2018 at 07:50:57AM -0500, David T. Lewis wrote:

> To handle all of the morphic projects, it would be better to do the
> cleanup when exiting the projects:
>
> MorphicProject>>finalExitActions: enteringProject
> world allMorphsDo: [:ea | ea removeProperty: #dropShadow].
> ...
>
> That will leave only the current project to worry about when saving
> the image. Of course, you do not want to to cleam up the morphs if
> the current world is a ControlManager, so it might look something
> like this:
>
> | world |
> (world := Project current world) isMorph
> ifTrue: [world allMorphsDo: [:ea | ea removeProperty: #dropShadow]]
>
> Dave
>
> On Tue, Feb 20, 2018 at 11:26:19AM +0100, Marcel Taeumel wrote:
> > We have to consider all open projects:
> >
> > Project allMorphicProjects do: [:project |
> > ?? ??project world allMorphsDo: [:morph |
> > ?? ?? ?? morph removeProperty: #dropShadow]].
> >
> > Yet, there can be hidden morphs. If we want to ignore those, we can stick with only cleaning up top-level morphs, which are usually the system windows:
> >
> > Project allMorphicProjects do: [:project |
> > ?? ??project world submorphsDo: [:morph |
> > ?? ?? ?? morph removeProperty: #dropShadow]].
> >
> > Best,
> > Marcel
> > Am 20.02.2018 10:57:08 schrieb Bob Arning :
> > Probably good enough might be
> > World allMorphsDo: [:ea | ea removeProperty: #dropShadow]
> > takes 0 ms for me
> >
> >
> > On 2/20/18 2:46 AM, Marcel Taeumel wrote:
> >
> > Hi Eliot,
> >
> > sure. Removing the potential drop shadow of all kinds of morphs takes time:
> >
> > Morph allSubInstancesDo: [:ea |??ea removeProperty: #dropShadow].
> >
> > About 3 seconds here in a quite clean image.
> >
> > SystemWindow allSubInstancesDo: [:ea |??ea removeProperty: #dropShadow].
> >
> > Works at 100 milliseconds.
> >
> > What has the biggest effect in your image?
> >
> > Best,
> > Marcel
> > Am 20.02.2018 02:51:06 schrieb Eliot Miranda [[hidden email]]:
> > Hi Marcel,
> >
> > ?? ?? I've finally worked out that the space overhead here comes from the dropShadow cache system in Morphs (in otherProperties in MorphExtensions).?? Would it be possible to arrange to flush all drop shadows in the current project when:
> > - exiting a project
> > - snapshotting
> > ?
> >
> > On Thu, Jan 18, 2018 at 3:24 PM, Eliot Miranda wrote:
> >
> > Hi All,
> >
> > ?? ?? I've been experiencing image save slowdowns recently and finally my work image reached 1.%Gb and I thought I better take a look:
> >
> > Sisyphus.Cog$ ls -lh SpurWork64.* save/SpurWork64-*
> > -rw-r--r--@ 1 eliot ??staff ?? ??28M Jan 18 12:47 SpurWork64.changes
> > -rw-r--r--@ 1 eliot ??staff ?? 1.6G Jan 18 12:48 SpurWork64.image
> > -rw-r--r--@ 1 eliot ??staff ?? ??28M Jan 18 12:03 save/SpurWork64-2018-01-18.changes
> > -rw-r--r--@ 1 eliot ??staff ?? 1.5G Jan 18 12:03 save/SpurWork64-2018-01-18.image
> >
> > I ran a space analysis and found that Bitmap and ByteArray were the top two, so I looked for large Bitmaps.?? I found three that fit this criterion:
> >
> >
> > ?? ?? Bitmap allInstances select: [:bm| bm size >= 1000000 and: [bm ~~ Display bits]]
> >
> > I inspected the three and did a chase pointers on one of them.?? As I did that suddenly
> > a) the inspector on the Array became empty (still an array but zero elements)
> > b) the progress bar for Downloading FlexibleVocabularies-who.NN appeared
> >
> > I interrupted this and did a very cursory stack examination. Some object had not understood isLiteral and from there what looked like an attempt to turn this stub into a real object caused FlexibleVocabularies-who.NN to start to download.
> >
> > I threw away the debugger, ran the GC and suddenly all my free space was back.?? So now on disc I have
> >
> > Sisyphus.Cog$ ls -lh SpurWork64.* save/SpurWork64-*
> > -rw-r--r--@ 1 eliot ??staff ?? ??28M Jan 18 15:17 SpurWork64.changes
> > -rw-r--r--@ 1 eliot ??staff ?? ??57M Jan 18 15:17 SpurWork64.image
> > -rw-r--r--@ 1 eliot ??staff ?? ??28M Jan 18 12:03 save/SpurWork64-2018-01-18.changes
> > -rw-r--r--@ 1 eliot ??staff ?? 1.5G Jan 18 12:03 save/SpurWork64-2018-01-18.image
> >
> > What is going on here??? There seems to be a very bad storage leak.?? Can we please discuss this??? This doesn't seem like healthy behaviour at all :-)
> >
> > _,,,^..^,,,_
> >
> > best,??Eliot
> >
> >
> >
> > --
> >
> > _,,,^..^,,,_
> >
> > best,??Eliot
> >
> >
>
> >
>
>




Reply | Threaded
Open this post in threaded view
|

Re: Serious garbage/storage leak issue with MCInfoProxy

marcel.taeumel
Hi Eliot,

okay. I added the clean-up to project switching and save-and/or-quit. All changes are in MorphicProject. See Morphic-mt.1397.

Best,
Marcel

Am 20.02.2018 14:14:01 schrieb Eliot Miranda <[hidden email]>:

Hi Marcel,


On Feb 20, 2018, at 5:00 AM, Marcel Taeumel <[hidden email]> wrote:

Hmm... I do not see value in cleaning up projects (to free memory) when the user just switches between them back and forth. Especially in the light of worlds-in-worlds or background projects, this seems like an unreasonable path to follow. We should not make too much assumptions about application code or user needs at this point. There could always be a "Oh, please render that background project into this file. Oh, why is it so slow at the moment..."

Well one might want to make the drop shadow caches responsive to some overall memory budget because, as I've observed, the overhead of the drop shadows can be many megabytes.  I saw an overhead that approached 1Gb!!

Cleaning up resources for save-and-quit, however, seems reasonable because it is at the environment's boundary and not some arbitrary in-image boundary.

Best,
Marcel

Am 20.02.2018 13:58:38 schrieb David T. Lewis <[hidden email]>:

This might also justify adding one new method to Project:

Project>>cleanUpForExit

Default implementation would be do nothing, and for MorphicProject it
would clean up the drop morphs. This would be called from #finalExitActions:
for ongoing cleanup, and when saving the image you would do this:

Project current cleanupForExit

Dave


On Tue, Feb 20, 2018 at 07:50:57AM -0500, David T. Lewis wrote:

> To handle all of the morphic projects, it would be better to do the
> cleanup when exiting the projects:
>
> MorphicProject>>finalExitActions: enteringProject
> world allMorphsDo: [:ea | ea removeProperty: #dropShadow].
> ...
>
> That will leave only the current project to worry about when saving
> the image. Of course, you do not want to to cleam up the morphs if
> the current world is a ControlManager, so it might look something
> like this:
>
> | world |
> (world := Project current world) isMorph
> ifTrue: [world allMorphsDo: [:ea | ea removeProperty: #dropShadow]]
>
> Dave
>
> On Tue, Feb 20, 2018 at 11:26:19AM +0100, Marcel Taeumel wrote:
> > We have to consider all open projects:
> >
> > Project allMorphicProjects do: [:project |
> > ?? ??project world allMorphsDo: [:morph |
> > ?? ?? ?? morph removeProperty: #dropShadow]].
> >
> > Yet, there can be hidden morphs. If we want to ignore those, we can stick with only cleaning up top-level morphs, which are usually the system windows:
> >
> > Project allMorphicProjects do: [:project |
> > ?? ??project world submorphsDo: [:morph |
> > ?? ?? ?? morph removeProperty: #dropShadow]].
> >
> > Best,
> > Marcel
> > Am 20.02.2018 10:57:08 schrieb Bob Arning :
> > Probably good enough might be
> > World allMorphsDo: [:ea | ea removeProperty: #dropShadow]
> > takes 0 ms for me
> >
> >
> > On 2/20/18 2:46 AM, Marcel Taeumel wrote:
> >
> > Hi Eliot,
> >
> > sure. Removing the potential drop shadow of all kinds of morphs takes time:
> >
> > Morph allSubInstancesDo: [:ea |??ea removeProperty: #dropShadow].
> >
> > About 3 seconds here in a quite clean image.
> >
> > SystemWindow allSubInstancesDo: [:ea |??ea removeProperty: #dropShadow].
> >
> > Works at 100 milliseconds.
> >
> > What has the biggest effect in your image?
> >
> > Best,
> > Marcel
> > Am 20.02.2018 02:51:06 schrieb Eliot Miranda [[hidden email]]:
> > Hi Marcel,
> >
> > ?? ?? I've finally worked out that the space overhead here comes from the dropShadow cache system in Morphs (in otherProperties in MorphExtensions).?? Would it be possible to arrange to flush all drop shadows in the current project when:
> > - exiting a project
> > - snapshotting
> > ?
> >
> > On Thu, Jan 18, 2018 at 3:24 PM, Eliot Miranda wrote:
> >
> > Hi All,
> >
> > ?? ?? I've been experiencing image save slowdowns recently and finally my work image reached 1.%Gb and I thought I better take a look:
> >
> > Sisyphus.Cog$ ls -lh SpurWork64.* save/SpurWork64-*
> > -rw-r--r--@ 1 eliot ??staff ?? ??28M Jan 18 12:47 SpurWork64.changes
> > -rw-r--r--@ 1 eliot ??staff ?? 1.6G Jan 18 12:48 SpurWork64.image
> > -rw-r--r--@ 1 eliot ??staff ?? ??28M Jan 18 12:03 save/SpurWork64-2018-01-18.changes
> > -rw-r--r--@ 1 eliot ??staff ?? 1.5G Jan 18 12:03 save/SpurWork64-2018-01-18.image
> >
> > I ran a space analysis and found that Bitmap and ByteArray were the top two, so I looked for large Bitmaps.?? I found three that fit this criterion:
> >
> >
> > ?? ?? Bitmap allInstances select: [:bm| bm size >= 1000000 and: [bm ~~ Display bits]]
> >
> > I inspected the three and did a chase pointers on one of them.?? As I did that suddenly
> > a) the inspector on the Array became empty (still an array but zero elements)
> > b) the progress bar for Downloading FlexibleVocabularies-who.NN appeared
> >
> > I interrupted this and did a very cursory stack examination. Some object had not understood isLiteral and from there what looked like an attempt to turn this stub into a real object caused FlexibleVocabularies-who.NN to start to download.
> >
> > I threw away the debugger, ran the GC and suddenly all my free space was back.?? So now on disc I have
> >
> > Sisyphus.Cog$ ls -lh SpurWork64.* save/SpurWork64-*
> > -rw-r--r--@ 1 eliot ??staff ?? ??28M Jan 18 15:17 SpurWork64.changes
> > -rw-r--r--@ 1 eliot ??staff ?? ??57M Jan 18 15:17 SpurWork64.image
> > -rw-r--r--@ 1 eliot ??staff ?? ??28M Jan 18 12:03 save/SpurWork64-2018-01-18.changes
> > -rw-r--r--@ 1 eliot ??staff ?? 1.5G Jan 18 12:03 save/SpurWork64-2018-01-18.image
> >
> > What is going on here??? There seems to be a very bad storage leak.?? Can we please discuss this??? This doesn't seem like healthy behaviour at all :-)
> >
> > _,,,^..^,,,_
> >
> > best,??Eliot
> >
> >
> >
> > --
> >
> > _,,,^..^,,,_
> >
> > best,??Eliot
> >
> >
>
> >
>
>




Reply | Threaded
Open this post in threaded view
|

Re: Serious garbage/storage leak issue with MCInfoProxy

Bob Arning-2

If...

- SystemWindows are holding lots of drop shadows

- the shadow is only shown for the frontmost window

- recreating the shadow only takes around 20ms

Then SystemWindow>>#passivate seems like a nice place to keep things tidy. Works fine for me.


On 2/20/18 11:21 AM, Marcel Taeumel wrote:
Hi Eliot,

okay. I added the clean-up to project switching and save-and/or-quit. All changes are in MorphicProject. See Morphic-mt.1397.

Best,
Marcel

Am 20.02.2018 14:14:01 schrieb Eliot Miranda [hidden email]:

Hi Marcel,


On Feb 20, 2018, at 5:00 AM, Marcel Taeumel <[hidden email]> wrote:

Hmm... I do not see value in cleaning up projects (to free memory) when the user just switches between them back and forth. Especially in the light of worlds-in-worlds or background projects, this seems like an unreasonable path to follow. We should not make too much assumptions about application code or user needs at this point. There could always be a "Oh, please render that background project into this file. Oh, why is it so slow at the moment..."

Well one might want to make the drop shadow caches responsive to some overall memory budget because, as I've observed, the overhead of the drop shadows can be many megabytes.  I saw an overhead that approached 1Gb!!

Cleaning up resources for save-and-quit, however, seems reasonable because it is at the environment's boundary and not some arbitrary in-image boundary.

Best,
Marcel

Am 20.02.2018 13:58:38 schrieb David T. Lewis <[hidden email]>:

This might also justify adding one new method to Project:

Project>>cleanUpForExit

Default implementation would be do nothing, and for MorphicProject it
would clean up the drop morphs. This would be called from #finalExitActions:
for ongoing cleanup, and when saving the image you would do this:

Project current cleanupForExit

Dave


On Tue, Feb 20, 2018 at 07:50:57AM -0500, David T. Lewis wrote:
> To handle all of the morphic projects, it would be better to do the
> cleanup when exiting the projects:
>
> MorphicProject>>finalExitActions: enteringProject
> world allMorphsDo: [:ea | ea removeProperty: #dropShadow].
> ...
>
> That will leave only the current project to worry about when saving
> the image. Of course, you do not want to to cleam up the morphs if
> the current world is a ControlManager, so it might look something
> like this:
>
> | world |
> (world := Project current world) isMorph
> ifTrue: [world allMorphsDo: [:ea | ea removeProperty: #dropShadow]]
>
> Dave
>
> On Tue, Feb 20, 2018 at 11:26:19AM +0100, Marcel Taeumel wrote:
> > We have to consider all open projects:
> >
> > Project allMorphicProjects do: [:project |
> > ?? ??project world allMorphsDo: [:morph |
> > ?? ?? ?? morph removeProperty: #dropShadow]].
> >
> > Yet, there can be hidden morphs. If we want to ignore those, we can stick with only cleaning up top-level morphs, which are usually the system windows:
> >
> > Project allMorphicProjects do: [:project |
> > ?? ??project world submorphsDo: [:morph |
> > ?? ?? ?? morph removeProperty: #dropShadow]].
> >
> > Best,
> > Marcel
> > Am 20.02.2018 10:57:08 schrieb Bob Arning :
> > Probably good enough might be
> > World allMorphsDo: [:ea | ea removeProperty: #dropShadow]
> > takes 0 ms for me
> >
> >
> > On 2/20/18 2:46 AM, Marcel Taeumel wrote:
> >
> > Hi Eliot,
> >
> > sure. Removing the potential drop shadow of all kinds of morphs takes time:
> >
> > Morph allSubInstancesDo: [:ea |??ea removeProperty: #dropShadow].
> >
> > About 3 seconds here in a quite clean image.
> >
> > SystemWindow allSubInstancesDo: [:ea |??ea removeProperty: #dropShadow].
> >
> > Works at 100 milliseconds.
> >
> > What has the biggest effect in your image?
> >
> > Best,
> > Marcel
> > Am 20.02.2018 02:51:06 schrieb Eliot Miranda [[hidden email]]:
> > Hi Marcel,
> >
> > ?? ?? I've finally worked out that the space overhead here comes from the dropShadow cache system in Morphs (in otherProperties in MorphExtensions).?? Would it be possible to arrange to flush all drop shadows in the current project when:
> > - exiting a project
> > - snapshotting
> > ?
> >
> > On Thu, Jan 18, 2018 at 3:24 PM, Eliot Miranda wrote:
> >
> > Hi All,
> >
> > ?? ?? I've been experiencing image save slowdowns recently and finally my work image reached 1.%Gb and I thought I better take a look:
> >
> > Sisyphus.Cog$ ls -lh SpurWork64.* save/SpurWork64-*
> > -rw-r--r--@ 1 eliot ??staff ?? ??28M Jan 18 12:47 SpurWork64.changes
> > -rw-r--r--@ 1 eliot ??staff ?? 1.6G Jan 18 12:48 SpurWork64.image
> > -rw-r--r--@ 1 eliot ??staff ?? ??28M Jan 18 12:03 save/SpurWork64-2018-01-18.changes
> > -rw-r--r--@ 1 eliot ??staff ?? 1.5G Jan 18 12:03 save/SpurWork64-2018-01-18.image
> >
> > I ran a space analysis and found that Bitmap and ByteArray were the top two, so I looked for large Bitmaps.?? I found three that fit this criterion:
> >
> >
> > ?? ?? Bitmap allInstances select: [:bm| bm size >= 1000000 and: [bm ~~ Display bits]]
> >
> > I inspected the three and did a chase pointers on one of them.?? As I did that suddenly
> > a) the inspector on the Array became empty (still an array but zero elements)
> > b) the progress bar for Downloading FlexibleVocabularies-who.NN appeared
> >
> > I interrupted this and did a very cursory stack examination. Some object had not understood isLiteral and from there what looked like an attempt to turn this stub into a real object caused FlexibleVocabularies-who.NN to start to download.
> >
> > I threw away the debugger, ran the GC and suddenly all my free space was back.?? So now on disc I have
> >
> > Sisyphus.Cog$ ls -lh SpurWork64.* save/SpurWork64-*
> > -rw-r--r--@ 1 eliot ??staff ?? ??28M Jan 18 15:17 SpurWork64.changes
> > -rw-r--r--@ 1 eliot ??staff ?? ??57M Jan 18 15:17 SpurWork64.image
> > -rw-r--r--@ 1 eliot ??staff ?? ??28M Jan 18 12:03 save/SpurWork64-2018-01-18.changes
> > -rw-r--r--@ 1 eliot ??staff ?? 1.5G Jan 18 12:03 save/SpurWork64-2018-01-18.image
> >
> > What is going on here??? There seems to be a very bad storage leak.?? Can we please discuss this??? This doesn't seem like healthy behaviour at all :-)
> >
> > _,,,^..^,,,_
> >
> > best,??Eliot
> >
> >
> >
> > --
> >
> > _,,,^..^,,,_
> >
> > best,??Eliot
> >
> >
>
> >
>
>





    



Reply | Threaded
Open this post in threaded view
|

Re: Serious garbage/storage leak issue with MCInfoProxy

marcel.taeumel
If you would put it in #passivate or #lookUnfocused, then the cache would not work for moving and re-focusing windows anymore. :-) I do not have more data to support my claims here. :-(

Updating the shadow for a 2516@1378 window takes 350 milliseconds on my, rather fast, machine.

Best,
Marcel

Am 20.02.2018 17:59:13 schrieb Bob Arning <[hidden email]>:

If...

- SystemWindows are holding lots of drop shadows

- the shadow is only shown for the frontmost window

- recreating the shadow only takes around 20ms

Then SystemWindow>>#passivate seems like a nice place to keep things tidy. Works fine for me.


On 2/20/18 11:21 AM, Marcel Taeumel wrote:
Hi Eliot,

okay. I added the clean-up to project switching and save-and/or-quit. All changes are in MorphicProject. See Morphic-mt.1397.

Best,
Marcel

Am 20.02.2018 14:14:01 schrieb Eliot Miranda [hidden email]:

Hi Marcel,


On Feb 20, 2018, at 5:00 AM, Marcel Taeumel <[hidden email]> wrote:

Hmm... I do not see value in cleaning up projects (to free memory) when the user just switches between them back and forth. Especially in the light of worlds-in-worlds or background projects, this seems like an unreasonable path to follow. We should not make too much assumptions about application code or user needs at this point. There could always be a "Oh, please render that background project into this file. Oh, why is it so slow at the moment..."

Well one might want to make the drop shadow caches responsive to some overall memory budget because, as I've observed, the overhead of the drop shadows can be many megabytes.  I saw an overhead that approached 1Gb!!

Cleaning up resources for save-and-quit, however, seems reasonable because it is at the environment's boundary and not some arbitrary in-image boundary.

Best,
Marcel

Am 20.02.2018 13:58:38 schrieb David T. Lewis <[hidden email]>:

This might also justify adding one new method to Project:

Project>>cleanUpForExit

Default implementation would be do nothing, and for MorphicProject it
would clean up the drop morphs. This would be called from #finalExitActions:
for ongoing cleanup, and when saving the image you would do this:

Project current cleanupForExit

Dave


On Tue, Feb 20, 2018 at 07:50:57AM -0500, David T. Lewis wrote:
> To handle all of the morphic projects, it would be better to do the
> cleanup when exiting the projects:
>
> MorphicProject>>finalExitActions: enteringProject
> world allMorphsDo: [:ea | ea removeProperty: #dropShadow].
> ...
>
> That will leave only the current project to worry about when saving
> the image. Of course, you do not want to to cleam up the morphs if
> the current world is a ControlManager, so it might look something
> like this:
>
> | world |
> (world := Project current world) isMorph
> ifTrue: [world allMorphsDo: [:ea | ea removeProperty: #dropShadow]]
>
> Dave
>
> On Tue, Feb 20, 2018 at 11:26:19AM +0100, Marcel Taeumel wrote:
> > We have to consider all open projects:
> >
> > Project allMorphicProjects do: [:project |
> > ?? ??project world allMorphsDo: [:morph |
> > ?? ?? ?? morph removeProperty: #dropShadow]].
> >
> > Yet, there can be hidden morphs. If we want to ignore those, we can stick with only cleaning up top-level morphs, which are usually the system windows:
> >
> > Project allMorphicProjects do: [:project |
> > ?? ??project world submorphsDo: [:morph |
> > ?? ?? ?? morph removeProperty: #dropShadow]].
> >
> > Best,
> > Marcel
> > Am 20.02.2018 10:57:08 schrieb Bob Arning :
> > Probably good enough might be
> > World allMorphsDo: [:ea | ea removeProperty: #dropShadow]
> > takes 0 ms for me
> >
> >
> > On 2/20/18 2:46 AM, Marcel Taeumel wrote:
> >
> > Hi Eliot,
> >
> > sure. Removing the potential drop shadow of all kinds of morphs takes time:
> >
> > Morph allSubInstancesDo: [:ea |??ea removeProperty: #dropShadow].
> >
> > About 3 seconds here in a quite clean image.
> >
> > SystemWindow allSubInstancesDo: [:ea |??ea removeProperty: #dropShadow].
> >
> > Works at 100 milliseconds.
> >
> > What has the biggest effect in your image?
> >
> > Best,
> > Marcel
> > Am 20.02.2018 02:51:06 schrieb Eliot Miranda [[hidden email]]:
> > Hi Marcel,
> >
> > ?? ?? I've finally worked out that the space overhead here comes from the dropShadow cache system in Morphs (in otherProperties in MorphExtensions).?? Would it be possible to arrange to flush all drop shadows in the current project when:
> > - exiting a project
> > - snapshotting
> > ?
> >
> > On Thu, Jan 18, 2018 at 3:24 PM, Eliot Miranda wrote:
> >
> > Hi All,
> >
> > ?? ?? I've been experiencing image save slowdowns recently and finally my work image reached 1.%Gb and I thought I better take a look:
> >
> > Sisyphus.Cog$ ls -lh SpurWork64.* save/SpurWork64-*
> > -rw-r--r--@ 1 eliot ??staff ?? ??28M Jan 18 12:47 SpurWork64.changes
> > -rw-r--r--@ 1 eliot ??staff ?? 1.6G Jan 18 12:48 SpurWork64.image
> > -rw-r--r--@ 1 eliot ??staff ?? ??28M Jan 18 12:03 save/SpurWork64-2018-01-18.changes
> > -rw-r--r--@ 1 eliot ??staff ?? 1.5G Jan 18 12:03 save/SpurWork64-2018-01-18.image
> >
> > I ran a space analysis and found that Bitmap and ByteArray were the top two, so I looked for large Bitmaps.?? I found three that fit this criterion:
> >
> >
> > ?? ?? Bitmap allInstances select: [:bm| bm size >= 1000000 and: [bm ~~ Display bits]]
> >
> > I inspected the three and did a chase pointers on one of them.?? As I did that suddenly
> > a) the inspector on the Array became empty (still an array but zero elements)
> > b) the progress bar for Downloading FlexibleVocabularies-who.NN appeared
> >
> > I interrupted this and did a very cursory stack examination. Some object had not understood isLiteral and from there what looked like an attempt to turn this stub into a real object caused FlexibleVocabularies-who.NN to start to download.
> >
> > I threw away the debugger, ran the GC and suddenly all my free space was back.?? So now on disc I have
> >
> > Sisyphus.Cog$ ls -lh SpurWork64.* save/SpurWork64-*
> > -rw-r--r--@ 1 eliot ??staff ?? ??28M Jan 18 15:17 SpurWork64.changes
> > -rw-r--r--@ 1 eliot ??staff ?? ??57M Jan 18 15:17 SpurWork64.image
> > -rw-r--r--@ 1 eliot ??staff ?? ??28M Jan 18 12:03 save/SpurWork64-2018-01-18.changes
> > -rw-r--r--@ 1 eliot ??staff ?? 1.5G Jan 18 12:03 save/SpurWork64-2018-01-18.image
> >
> > What is going on here??? There seems to be a very bad storage leak.?? Can we please discuss this??? This doesn't seem like healthy behaviour at all :-)
> >
> > _,,,^..^,,,_
> >
> > best,??Eliot
> >
> >
> >
> > --
> >
> > _,,,^..^,,,_
> >
> > best,??Eliot
> >
> >
>
> >
>
>





    



Reply | Threaded
Open this post in threaded view
|

Re: Serious garbage/storage leak issue with MCInfoProxy

timrowledge


> On 20-02-2018, at 9:06 AM, Marcel Taeumel <[hidden email]> wrote:
>
> If you would put it in #passivate or #lookUnfocused, then the cache would not work for moving and re-focusing windows anymore. :-) I do not have more data to support my claims here. :-(
>
> Updating the shadow for a 2516@1378 window takes 350 milliseconds on my, rather fast, machine.

Well therein lies the crux of the problem; why does it take so long and use so much memory? So far as I can see (on a 17650 image) the dropshadow form is the size of the entire window as expanded by the shadow offsets. That seems a bit excessive in size, build-time and display time. Surely the typical case would need 2 skinny Forms?


tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Useful random insult:- He has a tenuous grip on the obvious.



Reply | Threaded
Open this post in threaded view
|

Re: Serious garbage/storage leak issue with MCInfoProxy

timrowledge
With some playing around with shadow related preferences and theme options it looks like there are some rather oddinteractions buried in there.

Turn on soft shadows from Extras-'Themes & Colors' and then pick a window. In its halo menu pick 'drop shadow'-'shadow offset' and move the indicator around. If I move it to the ~20 pixels range I see a strange mix of soft and hard shadow not to mention some 'interesting' visuals if I try to get shadows to the left of the window.

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Fractured Idiom:- MERCI RIEN - Thanks for nothin'.



Reply | Threaded
Open this post in threaded view
|

Re: Serious garbage/storage leak issue with MCInfoProxy

marcel.taeumel
Just for the record: Hibernate/Unhibernate takes roughly the same time as to recompute a drop show for 2546@1392. It reduces the form's bits size to about 0.004 % because most of the shadow form are zeros. Thus, the would be only a small gain to choose "hibernate" over "discard".

Not sure about all the BitBlt magic, but we could split the shadow form into multiple forms to reduce memory load. Yet, last time I checked, it took much longer so blit many forms for the shadow instead of one.

Best,
Marcel

Am 20.02.2018 19:19:05 schrieb tim Rowledge <[hidden email]>:

With some playing around with shadow related preferences and theme options it looks like there are some rather oddinteractions buried in there.

Turn on soft shadows from Extras-'Themes & Colors' and then pick a window. In its halo menu pick 'drop shadow'-'shadow offset' and move the indicator around. If I move it to the ~20 pixels range I see a strange mix of soft and hard shadow not to mention some 'interesting' visuals if I try to get shadows to the left of the window.

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Fractured Idiom:- MERCI RIEN - Thanks for nothin'.





12