[vw7.1] Memory Leak Issue?
Locked
 
|
Hi all, I am working on an application which has a UI (using widget: TextEditor) to show the log string corresponding to a search criteria. I see that once I give a wide search criteria, the memory usage of the image increases to 500%. The implementation reads BOSS files from a predefined location, filters logs and displays it in the TextEditor. I took the ClassReport before and after the functionality and found that the main increase in memory is due to objects of ByteString and Array.The memory used go down to (approx) 300% of initial value If I do a manual gc from VisualLauncher or if I clear the TextEditor My questions are 1. Is it really a memory leak or an abnormal memory usage ? 2. Is there a way to nab down the position of memory leak/shoot-up ? (I used ClassReport and AllocationProfiler only to understand that low level classes might be the problem. But what I expected was to find at least some high-level classes so that I can understand the position of the leak) Any suggestions? Regards, Mani _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
Re: [vw7.1] Memory Leak Issue?
|
|
Mani,
Is all of your work done by the UI class? If so, I would suggest separating out the entry of the search criteria and the display of the results (in the UI class) from the work of using the search criteria to return the result (in a LoggerSearcher class?). This would allow you to run the search and retrieval without a UI with a fixed set of criteria and an expected result, measuring the memory usage as you make changes to your code/at different points in the execution of the search and retrieval. On 2/20/15 11:43 AM, mani kartha wrote:
_______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
|
|
In reply to this post by mani kartha
Mani, It is interesting that much of the memory does clean up, but not right away. Recent VW VMs have the ability to suspend GC. That feature is useful for callouts
to a database (so that weak object caches are the same after returning to VW). It is almost like GC was suspended until it was explicitly requested. The timing of tombstone and finalization [notification] from weak arrays can prevent a full GC on the first
pass. Your clue that clearing the TextEditor releases much of the memory suggests poor coding may be involved.
To find instances of "high-level classes"... Open an inspector on an object that you think should have been GC'ed. You can use #allInstances to find one of the instances to focus on. Select Object -> Inspect
Reference Paths from the inspector menu. You'll get an inspector on an OC of separate reference paths. Start with the last object on each reference path (the object you were inspecting) and work back toward the first position until you figure out what is a
poor reference. You could for example find the source to be some variable like "disconnectedSessions" that is rooted to a Global/Shared variable. I recall that a recently closed inspector would cache information until another inspector was opened. Sometimes
a menu action will be defined with a BlockClosure that is not clean, and that causes objects to linger. If you can't find the high-level objects then you might review object creation using MultiAllocationProfiler to see if object creation can either be reduced
or more strings retained as a flyweight string. I hope one of these ideas will help you find the problem. Paul Baumann From: [hidden email] [mailto:[hidden email]]
On Behalf Of mani kartha Hi all, I am working on an application which has a UI (using widget: TextEditor) to show the log string corresponding to a search criteria. I see that once I give
a wide search criteria, the memory usage of the image increases to 500%. The implementation reads BOSS files from a predefined location, filters logs and displays it in the TextEditor. I took the ClassReport before and after the functionality and found that
the main increase in memory is due to objects of ByteString and Array.The memory used go down to (approx) 300% of initial value If I do a manual gc from VisualLauncher or if I clear the TextEditor My questions are 1. Is it really a memory leak or an abnormal memory usage ? 2. Is there a way to nab down the position of memory leak/shoot-up ? (I used ClassReport and AllocationProfiler only to understand that low level classes
might be the problem. But what I expected was to find at least some high-level classes so that I can understand the position of the leak) Any suggestions? Regards, Mani This message may contain confidential information and is intended for specific recipients unless explicitly noted otherwise. If you have reason to believe you are not an intended recipient of this message, please delete it and notify the sender. This message may not represent the opinion of Intercontinental Exchange, Inc. (ICE), its subsidiaries or affiliates, and does not constitute a contract or guarantee. Unencrypted electronic mail is not secure and the recipient of this message is expected to provide safeguards from viruses and pursue alternate means of communication where privacy or a binding message is desired. _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
|
|
Hi all, thank you Paul & Tom ,your response inspired me to look more into the issue and I have narrowed my search to the following level. Given below is a replica of the implementation MyLogDisplayUI > showLogsforCriteria:aSearchCriteria | logObjects myStream | "section1:-getting log objects from BOSS files, which is done by a separate class instantiated prior to this method." "log objects are instances of a hierarchy of classes for example InformationLog,WarningLog,ErrorLog...etc" logObjects:=myLogRetriever getLogObjectsAccordingTo:aSearchCriteria "section2:- Iterating over the log objects to print it in to a stream" "there are two log levels-- #datailedLog & #summaryLog" myStream:=WriteStream on:String new. logObjects do:[:each|each printOn:myStream forLogLevel:#detailedLog] "section3:- displaying the contents of the stream in to the Text Editor " self myTextEditor value: myStream contents. Consider that initially my image consumed 90 Mb, I have 3 cases to detail case1: search criteria= getting 100 log objects after section1:97-98 Mb after section2:110-112 Mb after section3:110-112 Mb case2: search criteria= getting logs for 1 hour after section1:110-120 Mb after section2:350-380 Mb after section3:350-380 Mb case3: search criteria= getting logs for 1 day after section1:140-150 Mb after section2:470-490 Mb after section3:(error executing 'myStream contents'--could not create a copy of the myStream collection due to memory limitation. But i am able to terminate this error and the image is not crashed by this) If i try to run case3 with #summaryLog level in section2, the logs are displayed without memory issue. case4: search criteria= getting logs for 1 week after section1:300-350 Mb and then during section2 the image crashes GC at any of the level reclaim some amount of memory but it never goes back to < 100 Mb level unless i restart the image I did take a class report and attempted allInstances of suspicious classes but was unable to find a high-level object causing the issue. I only found thousands of ByteStrings and hundreds of Arrays all with similar kind of owners and reference paths, making it difficult for me to track them down manually. (by the way, taking Class reports reclaims some amount of memory?, I felt so) Please let me know if these details gives you some insight on this issue Thanks in advance Mani _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
|
|
>> (by the way, taking Class reports reclaims some amount of memory?, I felt so) Yes, #allInstances starts with GC. You may be interested in number of objects before the GC, and what is creating them. Use MultiAllocationProfiler to find
what is creating the objects. Instance creation will be in reading the BOSS file and also all the temporary strings created from #printString sends. The growth from the section1 BOSS reads
seems close to consumed memory between cases. Accumulation over time seems related to the BOSS read. Section2 produces the most objects, but they appear to be temporary objects that eventually cleanup. You might consider changing code to #printOn: instead of
#printString. You might consider pre-growing the string of your writestream to avoid incremental growth.
Focus on just one section at a time. Repeat section1 a few times to see if it accumulation happens and what you can do to force GC between executions.
Do you really need to use BOSS for log files? Most people just use text files. You might consider a log browser that starts with summaryLog and detailedLog
is read if a summaryLog is selected for display. You might implement and call #release methods to aid GC where practical. Regards, Paul Baumann From: [hidden email] [mailto:[hidden email]]
On Behalf Of mani kartha Hi all, thank you Paul & Tom ,your response inspired me to look more into the issue and I have narrowed my search to the following level. Given below is a replica of the implementation MyLogDisplayUI > showLogsforCriteria:aSearchCriteria | logObjects myStream | "section1:-getting log objects from BOSS files, which is done by a separate class instantiated prior to this method." "log objects are instances of a hierarchy of classes for example InformationLog,WarningLog,ErrorLog...etc" logObjects:=myLogRetriever getLogObjectsAccordingTo:aSearchCriteria "section2:- Iterating over the log objects to print it in to a stream" "there are two log levels-- #datailedLog & #summaryLog" myStream:=WriteStream on:String new. logObjects do:[:each|each printOn:myStream forLogLevel:#detailedLog] "section3:- displaying the contents of the stream in to the Text Editor " self myTextEditor value: myStream contents. Consider that initially my image consumed 90 Mb, I have 3 cases to detail case1: search criteria= getting 100 log objects after section1:97-98 Mb after section2:110-112 Mb after section3:110-112 Mb case2: search criteria= getting logs for 1 hour after section1:110-120 Mb after section2:350-380 Mb after section3:350-380 Mb case3: search criteria= getting logs for 1 day after section1:140-150 Mb after section2:470-490 Mb after section3:(error executing 'myStream contents'--could not create a copy of the myStream collection due to memory limitation. But i am able to terminate this
error and the image is not crashed by this) If i try to run case3 with #summaryLog level in section2, the logs are displayed without memory issue. case4: search criteria= getting logs for 1 week after section1:300-350 Mb and then during section2 the image crashes GC at any of the level reclaim some amount of memory but it never goes back to < 100 Mb level unless i restart the image I did take a class report and attempted allInstances of suspicious classes but was unable to find a high-level object causing the issue. I only found thousands of ByteStrings and hundreds of Arrays all with similar kind of owners and reference
paths, making it difficult for me to track them down manually. (by the way, taking Class reports reclaims some amount of memory?, I felt so) Please let me know if these details gives you some insight on this issue Thanks in advance Mani This message may contain confidential information and is intended for specific recipients unless explicitly noted otherwise. If you have reason to believe you are not an intended recipient of this message, please delete it and notify the sender. This message may not represent the opinion of Intercontinental Exchange, Inc. (ICE), its subsidiaries or affiliates, and does not constitute a contract or guarantee. Unencrypted electronic mail is not secure and the recipient of this message is expected to provide safeguards from viruses and pursue alternate means of communication where privacy or a binding message is desired. _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
| Free forum by Nabble | Edit this page |