Smalltalk › Instantiations

[8.5.2] Stack overflow memory anomaly

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

8 messages Options

Richard Sargent

[8.5.2] Stack overflow memory anomaly

Administrator

I ran a test with EsScanner that didn't work quite right (it never advanced to the next token). Since I was trying to collect the tokens from a source string, my OrderedCollection of tokens eventually grew so large that VA ran out of memory. No problem in itself.

However, I used the debugger to terminate the process, and saved the image. Again, that should not have been a problem either. The resulting saved image size was 16 GB! My previous image size was around the 75 MB range.

When I run the EsMemory... reports, they tell me I have approximately 75-80 MB of objects, as I expected. I do note that one of the EsMemorySegments is 15,700 MB in size (and 100% utilized). Running the garbage collector doesn't seem to clean it up. Restarting the image doesn't clean it up.

Is there any way to determine what that monster segment *thinks* its holding?
Is there any way to get it to give up and go away or to copy the real objects from it into another (new?) segment?

[Yes, I know I can (and probably will have to) reload my configuration and re-open the various text files, browsers, etc.]

--
You received this message because you are subscribed to the Google Groups "VA Smalltalk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at http://groups.google.com/group/va-smalltalk.
For more options, visit https://groups.google.com/d/optout.

Richard Sargent

Re: [8.5.2] Stack overflow memory anomaly

Administrator

On Wednesday, November 5, 2014 12:08:02 PM UTC-8, Richard Sargent wrote:

I ran a test with EsScanner that didn't work quite right (it never advanced to the next token). Since I was trying to collect the tokens from a source string, my OrderedCollection of tokens eventually grew so large that VA ran out of memory. No problem in itself.

However, I used the debugger to terminate the process, and saved the image. Again, that should not have been a problem either. The resulting saved image size was 16 GB! My previous image size was around the 75 MB range.

When I run the EsMemory... reports, they tell me I have approximately 75-80 MB of objects, as I expected. I do note that one of the EsMemorySegments is 15,700 MB in size (and 100% utilized). Running the garbage collector doesn't seem to clean it up. Restarting the image doesn't clean it up.

Well, surprise, surprise, surprise!
I looked in Object's Dependents class variable and noticed that the Code Assist tool had accumulated a number of entries. I brutally removed them all, saved my image, and lo, the image size dropped to 38 MB.

There are a couple of concerns here.
1) Code Assist is registering dependencies that are not being removed. This might be purely an 8.5.2 issue, but it might be more pervasive.
2) The memory utilization report could not explain the 16 GB of garbage.

Unfortunately, I did not expect the image size to shrink by much so I did not make a copy for further analysis.

Richard Sargent

Re: [8.5.2] Stack overflow memory anomaly

Administrator

In reply to this post by Richard Sargent

On Wednesday, November 5, 2014 12:08:02 PM UTC-8, Richard Sargent wrote:

I ran a test with EsScanner that didn't work quite right (it never advanced to the next token). Since I was trying to collect the tokens from a source string, my OrderedCollection of tokens eventually grew so large that VA ran out of memory. No problem in itself.

...
When I run the EsMemory... reports, they tell me I have approximately 75-80 MB of objects, as I expected. I do note that one of the EsMemorySegments is 15,700 MB in size (and 100% utilized). Running the garbage collector doesn't seem to clean it up. Restarting the image doesn't clean it up.

I have recreated this! The OrderedCollection contains 393 million references to the same string.
EsMemoryUseReport class>>#countAllMemory fails to report the existence of any Array instances!
(Array allInstances size ==> 163,119 - only one of which holds more than 1,000,000 elements)

So the two problems are:
1) Sometimes, the code assist dependents do not get removed.
2) EsMemoryUseReport lies.

Richard Sargent

Re: [8.5.2] Stack overflow memory anomaly

Administrator

On Thursday, November 6, 2014 9:47:48 AM UTC-8, Richard Sargent wrote:

On Wednesday, November 5, 2014 12:08:02 PM UTC-8, Richard Sargent wrote:
I ran a test with EsScanner that didn't work quite right (it never advanced to the next token). Since I was trying to collect the tokens from a source string, my OrderedCollection of tokens eventually grew so large that VA ran out of memory. No problem in itself.

...
When I run the EsMemory... reports, they tell me I have approximately 75-80 MB of objects, as I expected. I do note that one of the EsMemorySegments is 15,700 MB in size (and 100% utilized). Running the garbage collector doesn't seem to clean it up. Restarting the image doesn't clean it up.

I have recreated this! The OrderedCollection contains 393 million references to the same string.
EsMemoryUseReport class>>#countAllMemory fails to report the existence of any Array instances!
(Array allInstances size ==> 163,119 - only one of which holds more than 1,000,000 elements)

So the two problems are:
1) Sometimes, the code assist dependents do not get removed.
2) EsMemoryUseReport lies.

With regard to the first problem, when I completely closed the debugger in this recreation case, the dependencies were cleaned up and the image saved with the correct size. However, when I first encountered the problems, I had terminated the failing processes in the debugger, but retained one which I had been debugging. So, it could be that the code assist for the debugger is hanging on to more than it should. e.g., it shouldn't be responsible for holding anything to do with the terminated processes, such as the one which filled memory.

Carl Gundel-2

Re: [8.5.2] Stack overflow memory anomaly

In reply to this post by Richard Sargent

How did you manage to recreate it? Did you by any chance also experience any image hanging while all this was happening?

-Carl

On Thursday, November 6, 2014 12:47:48 PM UTC-5, Richard Sargent wrote:

On Wednesday, November 5, 2014 12:08:02 PM UTC-8, Richard Sargent wrote:
I ran a test with EsScanner that didn't work quite right (it never advanced to the next token). Since I was trying to collect the tokens from a source string, my OrderedCollection of tokens eventually grew so large that VA ran out of memory. No problem in itself.

...
When I run the EsMemory... reports, they tell me I have approximately 75-80 MB of objects, as I expected. I do note that one of the EsMemorySegments is 15,700 MB in size (and 100% utilized). Running the garbage collector doesn't seem to clean it up. Restarting the image doesn't clean it up.

I have recreated this! The OrderedCollection contains 393 million references to the same string.
EsMemoryUseReport class>>#countAllMemory fails to report the existence of any Array instances!
(Array allInstances size ==> 163,119 - only one of which holds more than 1,000,000 elements)

So the two problems are:
1) Sometimes, the code assist dependents do not get removed.
2) EsMemoryUseReport lies.

Richard Sargent

Re: [8.5.2] Stack overflow memory anomaly

Administrator

On Thursday, November 6, 2014 10:46:28 AM UTC-8, Carl Gundel wrote:

How did you manage to recreate it? Did you by any chance also experience any image hanging while all this was happening?

Hi Carl,

Once I understood what was happening, recreation was trivial. My actual code was more involved than the example below, but I believe it shows the salient characteristic.

| bloat |
bloat := OrderedCollection new.
[true] whileTrue: [bloat add: 'anything'].

Obviously, while this is running, the user interface is non-responsive. :-)
And once the debugger comes up with the out of memory error, you are somewhat limited in what you can do (since you can no longer allocate more memory).

Seth Berman

Re: [8.5.2] Stack overflow memory anomaly

In reply to this post by Richard Sargent

Hi Richard,

Thanks for your analysis. I'm going to try some more specific cases in the debugger to see if issues remain.

I came across this unfortunate property in the beginning of the 8.6.1 dev cycle and it should be fixed by the following case (see release notes of 8.6.1 when available).

And yes...I slapped my own hand for using 'Dependents':)
Case 53356: Fix memory retention of development windows, editors and code assistants.

I'll report back if I see any issues that will remain after 8.6.1.

-- Seth

On Thursday, November 6, 2014 12:56:12 PM UTC-5, Richard Sargent wrote:

On Thursday, November 6, 2014 9:47:48 AM UTC-8, Richard Sargent wrote:
On Wednesday, November 5, 2014 12:08:02 PM UTC-8, Richard Sargent wrote:
I ran a test with EsScanner that didn't work quite right (it never advanced to the next token). Since I was trying to collect the tokens from a source string, my OrderedCollection of tokens eventually grew so large that VA ran out of memory. No problem in itself.

...
When I run the EsMemory... reports, they tell me I have approximately 75-80 MB of objects, as I expected. I do note that one of the EsMemorySegments is 15,700 MB in size (and 100% utilized). Running the garbage collector doesn't seem to clean it up. Restarting the image doesn't clean it up.

I have recreated this! The OrderedCollection contains 393 million references to the same string.
EsMemoryUseReport class>>#countAllMemory fails to report the existence of any Array instances!
(Array allInstances size ==> 163,119 - only one of which holds more than 1,000,000 elements)

So the two problems are:
1) Sometimes, the code assist dependents do not get removed.
2) EsMemoryUseReport lies.

With regard to the first problem, when I completely closed the debugger in this recreation case, the dependencies were cleaned up and the image saved with the correct size. However, when I first encountered the problems, I had terminated the failing processes in the debugger, but retained one which I had been debugging. So, it could be that the code assist for the debugger is hanging on to more than it should. e.g., it shouldn't be responsible for holding anything to do with the terminated processes, such as the one which filled memory.

Richard Sargent

Re: [8.5.2] Stack overflow memory anomaly

Administrator

On Monday, November 10, 2014 6:05:30 AM UTC-8, Seth Berman wrote:

Hi Richard,

Thanks for your analysis. I'm going to try some more specific cases in the debugger to see if issues remain.
I came across this unfortunate property in the beginning of the 8.6.1 dev cycle and it should be fixed by the following case (see release notes of 8.6.1 when available).
And yes...I slapped my own hand for using 'Dependents':)
Case 53356: Fix memory retention of development windows, editors and code assistants.

I'll report back if I see any issues that will remain after 8.6.1.

Thanks, Seth.

Are you in a position to "sling a date" our here for when we should expect 8.6.1? The last word I have seen was "October". :-)
(I've been in the business long enough to know all dates are targets!)