Smalltalk › Squeak › Squeak VM

Too many full GCs in Cog/Spur ... while just drawing GUI ...

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

27 messages Options

Levente Uzonyi

Re: Too many full GCs in Cog/Spur ... while just drawing GUI ...

Interesting. I can reproduce this, but I blame it on
#primitiveGarbageCollect.

"Primitive. Reclaims all garbage and answers the size of the largest free chunk in old space.."

I suspect that the size of the largest free chunk is limited by the value
of VM parameter 25. If you evaluate

Smalltalk vmParameterAt: 25 put: 40000016.

Then OutOfMemory will not be signaled. I can't tell why it works without
my changes. Eliot? :)

Levente

Eliot Miranda-2

Re: Too many full GCs in Cog/Spur ... while just drawing GUI ...

In reply to this post by Bert Freudenberg

On Fri, Apr 8, 2016 at 2:57 PM, Bert Freudenberg <[hidden email]> wrote:

On 08.04.2016, at 20:00, Eliot Miranda <[hidden email]> wrote:

On Fri, Apr 8, 2016 at 9:29 AM, Levente Uzonyi <[hidden email]> wrote:
So I suggest we should use both changes if it has no negative side effects. Eliot? :)

+1. I had hoped that scavenging would be run automatically, but this won't happen with huge allocations. For small allocations, when eden is full, the machine code new primitive will set the "needs scavenge" flag, when #handleFailingBasicNew: runs the scavenger will run, and so there is no need to do Smalltalk garbageCollectMost, because that has happened implicitly. But for huge allocations I think the code doesn't set the scavenge flag, it merely fails the primitive. But I need to check this.

What do we prefer, having the machine code for new always set the "needs scavenge" flag if an allocation failed because there was no room, or have #handleFailingBasicNew: et al calls Smalltalk garbageCollectMost explicitly?

IMHO there’s no advantage to rely on the image to do that. I’d prefer handling it in the VM.

When I saw Bert's message above, denying that basicNew: ever fails, I immediately repeated Bert's experiment, /knowing/ that basicNew /does/ fail. But to my surprise my experiment revealed Bert's result, that basicNew does not appear to fail :-).

Now that’s a relief ;)

Of course the gotcher is that the backward branch in the loop over allocation checks for events, and runs the scavenger, so a simple loop never shows failures. It's difficult to make it fail, but fail it will :-)

Yeah. So more smaller allocations will not make it fail, but few bigger ones does.

Right.

When exactly is a full GC triggered in the current scheme?

When handleFailingBasicNew: fails to allocate. Here's the code:

Behavior>>basicNew

"Primitive. Answer an instance of the receiver (which is a class) with no

indexable variables. Fail if the class is indexable. Essential. See Object

documentation whatIsAPrimitive.

If the primitive fails because space is low then the scavenger will run

before the method is activated. Check that space was low and retry

via handleFailingBasicNew if so."

<primitive: 70 error: ec>

ec == #'insufficient object memory' ifTrue:

[^self handleFailingBasicNew].

self isVariable ifTrue: [^self basicNew: 0].

self primitiveFailed

handleFailingBasicNew

"handleFailingBasicNew gets sent after basicNew has failed and allowed

a scavenging garbage collection to occur. The scavenging collection

will have happened as the VM is activating the (failing) basicNew. If

handleFailingBasicNew fails then the scavenge failed to reclaim sufficient

space and a global garbage collection is required. Retry after garbage

collecting and growing memory if necessary.

Primitive. Answer an instance of this class with the number of indexable

variables specified by the argument, sizeRequested. Fail if this class is not

indexable or if the argument is not a positive Integer, or if there is not

enough memory available. Essential. See Object documentation whatIsAPrimitive."

<primitive: 70>

Smalltalk garbageCollect < 1048576 ifTrue:

[Smalltalk growMemoryByAtLeast: 1048576].

^self handleFailingFailingBasicNew "retry after global garbage collect"

handleFailingFailingBasicNew

"This basicNew gets sent after handleFailingBasicNew: has done a full

garbage collection and possibly grown memory. If this basicNew fails

then the system really is low on space, so raise the OutOfMemory signal.

Primitive. Answer an instance of this class with the number of indexable

variables specified by the argument, sizeRequested. Fail if this class is not

indexable or if the argument is not a positive Integer, or if there is not

enough memory available. Essential. See Object documentation whatIsAPrimitive."

<primitive: 70>

"space must be low"

OutOfMemory signal.

^self basicNew "retry if user proceeds"

So if one does SomeClass basicNew and this fails with the error code #'insufficient object memory' it did so because the VM is (at least temporarily) out of memory. If the primitive invoked by basicNew sets the "scavengeNeeded" flag, then by the time we get to the "^self handleFailingBasicNew" send in basicNew, the scavenger will have run, because events are responded to when a method builds a frame, which basicNew does as it fails. So basicNew sends handleFailingBasicNew to retry the allocation, assuming the scavenger has run.

So if handleFailingBasicNew fails it means there's not enough room in newSpace or in oldSpace. Therefore it runs Smalltalk garbageCollect to reclaim space in oldSpace, and then invokes handleFailingFailingBasicNew to retry the allocation a third time. If handleFailingFailingBasicNew it means we're really out of space and so it raises the OutOfMemory exception.

The code is similar for basicNew:.

- Bert -

_,,,^..^,,,_

best, Eliot

timrowledge

Re: Too many full GCs in Cog/Spur ... while just drawing GUI ...

>
> Yeah. So more smaller allocations will not make it fail, but few bigger ones does.
>
> Right.

And in #imageForm the first thing it does is make a damn big Form - in this example the same size as the Display, and on my Pi3 that means 5 and a half megabytes, each time around. Of course it fills up memory doing 125 runs of imageForm! And that is ignoring any other large Forms created by FormCanvas uses.

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Document the code? Why do you think they call it "code?"

marcel.taeumel

Re: Too many full GCs in Cog/Spur ... while just drawing GUI ...

Hi Tim,

I noticed the sound glitches while resizing system windows. Then I got curious whether such trivial GUI operations should trigger full GCs so frequently. This "ActiveWorld imageForm" felt close enough to the problem.

So, yes, "just redrawing the GUI" via "ActiveWorld changed; displayWorldSafely" does not trigger full GCs at all. Sorry, for the unclear problem description. :-)

This does not even trigger the full GCs:

x := PluggableSystemWindow someInstance.
delay := Delay forMilliseconds: 50.
[
1 to: 50 do: [:ea |
x extent: (ea * 8) asPoint.
ActiveWorld displayWorldSafely.
delay wait]
] repeat.

Hmmm... What is the difference between resizing the window with the mouse and doing it via #extent:? Ha! It could be the shadow when the window is active:

x := PluggableSystemWindow someInstance.
delay := Delay forMilliseconds: 50.
x activate. "for the shadow."
[

1 to: 50 do: [:ea |
x extent: (ea * 10) asPoint.
ActiveWorld displayWorldSafely.
delay wait]
] repeat

An computing the shadow involves creating forms for the shadow cache. Hmmm... So yes, it is about these big forms create/deletion, which worked way better in non-Spur than it does in Spur. :-)

So, let's deactivate the shadow via "menuAppearance3d" and try again... no full GCs. Because Morph >> #updateDropShadowCache is not called and no forms are created.

Best,
Marcel

timrowledge

Re: Too many full GCs in Cog/Spur ... while just drawing GUI ...

> On 08-04-2016, at 11:30 PM, marcel.taeumel <[hidden email]> wrote:
>
> Hmmm... What is the difference between resizing the window with the mouse
> and doing it via #extent:? Ha! It could be the shadow when the window is
> active:

Yup. It’s so easy to forget the size of bitmaps. Those of us that grew up in the days of 386 PC/68k Macs tend to think of them as big things to be carefully handled - unless creeping senility has… whatever. Those of you that have grown up in a world of multi-core, multi-GHz CPUs with 1/2/4Gb of ram generally don’t worry about memory size at all. Both approaches can be very wrong under the right circumstances.

Scratch used to have a glorious example of ‘being oblivious to the size of things’ for drawing a border around the scripts. It involved rendering the entire script onto a new form with transparency (which was a recursive thing with many new Forms involved) and then rendering that into black and white to make a mask, then doing something with that in all 8 directions to make a sort of splodged-out shape then … I forget. It was horribly expensive in both memory and time, so eventually I just stopped it doing that. Which is what the ‘special handling of a single submorph that wants to do its own thing’ section of HandMorph>fullDrawOn: is there for.

There is a reasonably obvious special case for rectangular things like typical windows in that one could make smaller shadow morphs corresponding to the two edges casting shadows and then simply display them alongside the window as it moves. This would save quite a bit of memory and time.

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Strange OpCodes: GSI: Garble Subsequent Instruction

Bert Freudenberg

Re: Too many full GCs in Cog/Spur ... while just drawing GUI ...

In reply to this post by Eliot Miranda-2

On 09.04.2016, at 01:48, Eliot Miranda <[hidden email]> wrote:

On Fri, Apr 8, 2016 at 2:57 PM, Bert Freudenberg <[hidden email]> wrote:

When exactly is a full GC triggered in the current scheme?

When handleFailingBasicNew: fails to allocate.

Well I understand that code, but I was asking about the VM-internal triggers.

I just changed both basicNew: and basicNew like this:

basicNew: sizeRequested

<primitive: 71 error: ec>

BasicNewFailures := BasicNewFailures + 1.

(ec == #'insufficient object memory' or: [ec == #'bad argument']) ifTrue:

[^self handleFailingBasicNew: sizeRequested].

self isVariable ifFalse:

[self error: self printString, ' cannot have variable sized instances'].

self primitiveFailed

And ran this:

before := {Smalltalk vmParameterAt: 7. BasicNewFailures}.

1 to: 100000 do: [:i | Array new: 10000].

after := {Smalltalk vmParameterAt: 7. BasicNewFailures}.

{before. after}

==> #(#(111 0) #(129 0))

This looks like there were 18 full GCs even though basicNew never failed. So there must be some other path to trigger a full GC, right?

- Bert -

smime.p7s (5K) Download Attachment

Levente Uzonyi

Re: Too many full GCs in Cog/Spur ... while just drawing GUI ...

Try allocating non-pointer objects.

Levente