Multiple entries for same class in out of memory crash report .csv file

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Multiple entries for same class in out of memory crash report .csv file

GLASS mailing list
Hi guys,

Under 3.3.3 when an gem out of memory crashes happens, a topazXXX.csv is generated with the instance count. However, I found out that there are many classes that are repeated (have more than 1 row). And sometimes with different values on the columns! So I am not sure if they appears more than once but only one is the "latest" of if the real value should be the sum of all appearences. In either case, I don't think they should appear more than once.

If this is the first time you hear about this problem, please let me know and I send you an .csv example.


_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|

Re: Multiple entries for same class in out of memory crash report .csv file

GLASS mailing list

Mariano,

Is it possible that you have multiple versions of that class?


On 06/28/2017 08:11 AM, Mariano Martinez Peck via Glass wrote:
Hi guys,

Under 3.3.3 when an gem out of memory crashes happens, a topazXXX.csv is generated with the instance count. However, I found out that there are many classes that are repeated (have more than 1 row). And sometimes with different values on the columns! So I am not sure if they appears more than once but only one is the "latest" of if the real value should be the sum of all appearences. In either case, I don't think they should appear more than once.

If this is the first time you hear about this problem, please let me know and I send you an .csv example.

Thanks, 

--


_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass


_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|

Re: Multiple entries for same class in out of memory crash report .csv file

GLASS mailing list


On Wed, Jun 28, 2017 at 2:42 PM, Dale Henrichs via Glass <[hidden email]> wrote:

Mariano,

Is it possible that you have multiple versions of that class?



Ok, that was a good question. I didn't even think about that (I wish the class history number / index would be printed together with the class name in that case).  But no, I see repeated classes like Array, String, LogEntry etc which clearly do not have classHistory > 1.

Cheers,


 
On 06/28/2017 08:11 AM, Mariano Martinez Peck via Glass wrote:
Hi guys,

Under 3.3.3 when an gem out of memory crashes happens, a topazXXX.csv is generated with the instance count. However, I found out that there are many classes that are repeated (have more than 1 row). And sometimes with different values on the columns! So I am not sure if they appears more than once but only one is the "latest" of if the real value should be the sum of all appearences. In either case, I don't think they should appear more than once.

If this is the first time you hear about this problem, please let me know and I send you an .csv example.

Thanks, 

--


_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass


_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass




--

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|

Re: Multiple entries for same class in out of memory crash report .csv file

GLASS mailing list

Mariano,

Here's Allen's response:

The csv file includes data on each region of temp obj memory, and always includes oops.


topaz> login
successful login
topaz 1> run
| a | a := { } .
[ true ] whileTrue:[ a add: (String new:1000)]
%
*********** Fatal error 4067 OutOfMemory *****
...
Wrote object memory usage to topaz32098.csv
ERROR 4067 , VM temporary object memory is full
, old space overflow
topaz> quit


% egrep -e '^String|Instances|className'  topaz32098.csv
Instances counts for generation all
className,classOop,numberOfInstances,totalSizeInBytes
String,74753,39535,40268208
StringKeyValueDictionary,79873,2,272
Instances counts for generation new
className,classOop,numberOfInstances,totalSizeInBytes
String,74753,2599,2661376
Instances counts for generation pom
className,classOop,numberOfInstances,totalSizeInBytes
String,74753,2,72
StringKeyValueDictionary,79873,2,272
Instances counts for generation old
className,classOop,numberOfInstances,totalSizeInBytes
String,74753,36934,37606760
Instances counts for generation perm
className,classOop,numberOfInstances,totalSizeInBytes
Instances counts for generation code
className,classOop,numberOfInstances,totalSizeInBytes



On 06/28/2017 10:51 AM, Mariano Martinez Peck wrote:


On Wed, Jun 28, 2017 at 2:42 PM, Dale Henrichs via Glass <[hidden email]> wrote:

Mariano,

Is it possible that you have multiple versions of that class?



Ok, that was a good question. I didn't even think about that (I wish the class history number / index would be printed together with the class name in that case).  But no, I see repeated classes like Array, String, LogEntry etc which clearly do not have classHistory > 1.

Cheers,


 
On 06/28/2017 08:11 AM, Mariano Martinez Peck via Glass wrote:
Hi guys,

Under 3.3.3 when an gem out of memory crashes happens, a topazXXX.csv is generated with the instance count. However, I found out that there are many classes that are repeated (have more than 1 row). And sometimes with different values on the columns! So I am not sure if they appears more than once but only one is the "latest" of if the real value should be the sum of all appearences. In either case, I don't think they should appear more than once.

If this is the first time you hear about this problem, please let me know and I send you an .csv example.

Thanks, 

--


_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
_______________________________________________ Glass mailing list [hidden email] http://lists.gemtalksystems.com/mailman/listinfo/glass
--

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|

Re: Multiple entries for same class in out of memory crash report .csv file

GLASS mailing list


On Wed, Jun 28, 2017 at 4:17 PM, Dale Henrichs <[hidden email]> wrote:

Mariano,

Here's Allen's response:

The csv file includes data on each region of temp obj memory, and always includes oops.

 
topaz> login
successful login
topaz 1> run
| a | a := { } .
[ true ] whileTrue:[ a add: (String new:1000)]
%
*********** Fatal error 4067 OutOfMemory *****
...
Wrote object memory usage to topaz32098.csv
ERROR 4067 , VM temporary object memory is full
, old space overflow
topaz> quit


% egrep -e '^String|Instances|className'  topaz32098.csv
Instances counts for generation all
className,classOop,numberOfInstances,totalSizeInBytes
String,74753,39535,40268208
StringKeyValueDictionary,79873,2,272
Instances counts for generation new
className,classOop,numberOfInstances,totalSizeInBytes
String,74753,2599,2661376
Instances counts for generation pom
className,classOop,numberOfInstances,totalSizeInBytes
String,74753,2,72
StringKeyValueDictionary,79873,2,272
Instances counts for generation old
className,classOop,numberOfInstances,totalSizeInBytes
String,74753,36934,37606760
Instances counts for generation perm
className,classOop,numberOfInstances,totalSizeInBytes
Instances counts for generation code
className,classOop,numberOfInstances,totalSizeInBytes




Ohh I see now.  Ok, honestly, now I remember I saw that when I saw the "original" file. 
The "problem" is that obviously, when you got  a out of memory, you are interested in the guys taking the most memory. So, what i do is to take the file, open it with Numbers and sort by either totalSizeInBytes or numberOfInstances, hence the "duplicates". But with current state is hard to analyze because same class can have multiple rows and so i have to "mentally" sum and "reorder" in my mind.

I guess I will write an utility class that receives the original csv file and outputs desired files:

FaOutOfMemoryCsvConverter convertFrom: 'topazXXX.csv' tallingAllMemoryRegions: true sortByColumnKey: #numberOfInstances to: 'adapted.csv'

groupAllRegions if true, would sum/tally all regions together. If false, then each region will have a separate space in the file separated by divider rows. 

#numberOfInstances would sort the resulting csv (wether grouping or not was applied).

That way I can choose how to group and how to sort. 

Does this sound as a good idea? 

Cheers,


 

On 06/28/2017 10:51 AM, Mariano Martinez Peck wrote:


On Wed, Jun 28, 2017 at 2:42 PM, Dale Henrichs via Glass <[hidden email]> wrote:

Mariano,

Is it possible that you have multiple versions of that class?



Ok, that was a good question. I didn't even think about that (I wish the class history number / index would be printed together with the class name in that case).  But no, I see repeated classes like Array, String, LogEntry etc which clearly do not have classHistory > 1.

Cheers,


 
On 06/28/2017 08:11 AM, Mariano Martinez Peck via Glass wrote:
Hi guys,

Under 3.3.3 when an gem out of memory crashes happens, a topazXXX.csv is generated with the instance count. However, I found out that there are many classes that are repeated (have more than 1 row). And sometimes with different values on the columns! So I am not sure if they appears more than once but only one is the "latest" of if the real value should be the sum of all appearences. In either case, I don't think they should appear more than once.

If this is the first time you hear about this problem, please let me know and I send you an .csv example.

Thanks, 

--


_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
_______________________________________________ Glass mailing list [hidden email] http://lists.gemtalksystems.com/mailman/listinfo/glass
--



--

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|

Re: Multiple entries for same class in out of memory crash report .csv file

GLASS mailing list
In private email, Allen suggested the following:

----
The section of the file  beginning with   "Instances counts for generation all"
will have the total for all of temporary object memory.  So in most cases you
could just ignore the other "Instance counts for ..." sections.

In rare cases, when a generation other than "old"  filled up first you
might need to look at other  parts of the  .csv  file .

----

Which  is a very nice idea. So... if you only care about "generation all" (as I do most of the cases) and you want to open the CSV with a excel like (to sort etc), then you can do this one-liner:

 tail -n +2 topaz11203.csv | sed '/Instances counts for generation new/q' | sed '$d' > topazProcessed.csv                                                     
That outputs a processed csv with just the "all generation" contents. 

Thank you very much for your answers.
 



On Wed, Jun 28, 2017 at 4:33 PM, Mariano Martinez Peck <[hidden email]> wrote:


On Wed, Jun 28, 2017 at 4:17 PM, Dale Henrichs <[hidden email]> wrote:

Mariano,

Here's Allen's response:

The csv file includes data on each region of temp obj memory, and always includes oops.

 
topaz> login
successful login
topaz 1> run
| a | a := { } .
[ true ] whileTrue:[ a add: (String new:1000)]
%
*********** Fatal error 4067 OutOfMemory *****
...
Wrote object memory usage to topaz32098.csv
ERROR 4067 , VM temporary object memory is full
, old space overflow
topaz> quit


% egrep -e '^String|Instances|className'  topaz32098.csv
Instances counts for generation all
className,classOop,numberOfInstances,totalSizeInBytes
String,74753,39535,40268208
StringKeyValueDictionary,79873,2,272
Instances counts for generation new
className,classOop,numberOfInstances,totalSizeInBytes
String,74753,2599,2661376
Instances counts for generation pom
className,classOop,numberOfInstances,totalSizeInBytes
String,74753,2,72
StringKeyValueDictionary,79873,2,272
Instances counts for generation old
className,classOop,numberOfInstances,totalSizeInBytes
String,74753,36934,37606760
Instances counts for generation perm
className,classOop,numberOfInstances,totalSizeInBytes
Instances counts for generation code
className,classOop,numberOfInstances,totalSizeInBytes




Ohh I see now.  Ok, honestly, now I remember I saw that when I saw the "original" file. 
The "problem" is that obviously, when you got  a out of memory, you are interested in the guys taking the most memory. So, what i do is to take the file, open it with Numbers and sort by either totalSizeInBytes or numberOfInstances, hence the "duplicates". But with current state is hard to analyze because same class can have multiple rows and so i have to "mentally" sum and "reorder" in my mind.

I guess I will write an utility class that receives the original csv file and outputs desired files:

FaOutOfMemoryCsvConverter convertFrom: 'topazXXX.csv' tallingAllMemoryRegions: true sortByColumnKey: #numberOfInstances to: 'adapted.csv'

groupAllRegions if true, would sum/tally all regions together. If false, then each region will have a separate space in the file separated by divider rows. 

#numberOfInstances would sort the resulting csv (wether grouping or not was applied).

That way I can choose how to group and how to sort. 

Does this sound as a good idea? 

Cheers,


 

On 06/28/2017 10:51 AM, Mariano Martinez Peck wrote:


On Wed, Jun 28, 2017 at 2:42 PM, Dale Henrichs via Glass <[hidden email]> wrote:

Mariano,

Is it possible that you have multiple versions of that class?



Ok, that was a good question. I didn't even think about that (I wish the class history number / index would be printed together with the class name in that case).  But no, I see repeated classes like Array, String, LogEntry etc which clearly do not have classHistory > 1.

Cheers,


 
On 06/28/2017 08:11 AM, Mariano Martinez Peck via Glass wrote:
Hi guys,

Under 3.3.3 when an gem out of memory crashes happens, a topazXXX.csv is generated with the instance count. However, I found out that there are many classes that are repeated (have more than 1 row). And sometimes with different values on the columns! So I am not sure if they appears more than once but only one is the "latest" of if the real value should be the sum of all appearences. In either case, I don't think they should appear more than once.

If this is the first time you hear about this problem, please let me know and I send you an .csv example.

Thanks, 

--


_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
_______________________________________________ Glass mailing list [hidden email] http://lists.gemtalksystems.com/mailman/listinfo/glass
--



--



--

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass