Moose 5.1 image getting really sluggish after graphing a bigger dataset
Locked
Moose 5.1 image getting really sluggish after graphing a bigger dataset
 
|
Hi everyone,
after having access to a machine running OS X 10.10.2 I started to give Moose 5.1 a try for some initial data analysis. I used Pharo Launcher (build #24) and downloaded the Moose image with buildnumber 304. The task is to read a CSV log file containing around 130000 entries, each containing a timestamp, a cpu usage number, and some other information. Everything works as expected, but the responsiveness of the image drops dramatically after executing the script in Playground with Cmd+Shift+g. After evaluating the script it takes several seconds to get a response to any action such as opening the world menu or switching tabs in the playground. Is this because I am doing something very inefficient (most probably) or does this problem occur more often? Here is the script I am using: cpuReader := NeoCSVReader on: (cpuFile readStream). cpuReader separator: $|. cpuHeader := cpuReader readHeader. cpuData := cpuReader upToEnd. g := RTGrapher new. g extent: 300@200. ds := RTDataSet new. ds points: (cpuData collect: [ :each | | timestamp cpuUsage | timestamp := (each at: 1) asNumber. cpuUsage := (each at: 4) asNumber. Array braceWith: timestamp with: cpuUsage]). ds x: #first. ds y: #second. ds connectColor: Color blue. g add: ds. g addDecorator: RTHorizontalTickLineDecorator new. g addDecorator: RTVerticalTickLineDecorator new. g build. g view canvas. I also noticed that the 'Meta' representation of an object looks very crammed since it does not use the full size available in the tab. Should issues like this be reported somewhere else? Thanks for taking a look at this, Manfred _______________________________________________ Moose-dev mailing list [hidden email] https://www.iam.unibe.ch/mailman/listinfo/moose-dev |
Re: Moose 5.1 image getting really sluggish after graphing a bigger dataset
|
|
Hi Manfred,
I can only help you with the NeoCSV & IO part. Here is how I would do it: [ 'manfred.csv' asFileReference writeStreamDo: [ :out | (NeoCSVWriter on: (ZnBufferedWriteStream on: out)) in: [ :writer | writer separator: $|; writeHeader: #(timestamp two three cpu); fieldWriter: #raw. 130000 timesRepeat: [ writer nextPut: { DateAndTime now asUnixTime. 999 atRandom. 999 atRandom. 999 atRandom } ] ] ]. ] timeToRun. "0:00:00:01.917" [ 'manfred.csv' asFileReference readStreamDo: [ :in | (NeoCSVReader on: (ZnBufferedReadStream on: in)) in: [ :reader | reader separator: $|; readHeader. reader addIntegerField; addIgnoredField; addIgnoredField; addIntegerField. reader upToEnd ] ]. ] timeToRun. "0:00:00:01.064" I think that is acceptable for a 3Mb file. HTH, Sven > On 16 Mar 2015, at 17:48, Manfred Kröhnert <[hidden email]> wrote: > > Hi everyone, > > after having access to a machine running OS X 10.10.2 I started to give Moose 5.1 a try for some initial data analysis. > I used Pharo Launcher (build #24) and downloaded the Moose image with buildnumber 304. > > The task is to read a CSV log file containing around 130000 entries, each containing a timestamp, a cpu usage number, and some other information. > Everything works as expected, but the responsiveness of the image drops dramatically after executing the script in Playground with Cmd+Shift+g. > After evaluating the script it takes several seconds to get a response to any action such as opening the world menu or switching tabs in the playground. > > Is this because I am doing something very inefficient (most probably) or does this problem occur more often? > Here is the script I am using: > > > cpuReader := NeoCSVReader on: (cpuFile readStream). > cpuReader separator: $|. > cpuHeader := cpuReader readHeader. > cpuData := cpuReader upToEnd. > > g := RTGrapher new. > g extent: 300@200. > > ds := RTDataSet new. > ds points: (cpuData collect: [ :each | > | timestamp cpuUsage | > timestamp := (each at: 1) asNumber. > cpuUsage := (each at: 4) asNumber. > Array braceWith: timestamp with: cpuUsage]). > ds x: #first. > ds y: #second. > ds connectColor: Color blue. > > g add: ds. > g addDecorator: RTHorizontalTickLineDecorator new. > g addDecorator: RTVerticalTickLineDecorator new. > g build. > g view canvas. > > > I also noticed that the 'Meta' representation of an object looks very crammed since it does not use the full size available in the tab. > Should issues like this be reported somewhere else? > > Thanks for taking a look at this, > Manfred > _______________________________________________ > Moose-dev mailing list > [hidden email] > https://www.iam.unibe.ch/mailman/listinfo/moose-dev _______________________________________________ Moose-dev mailing list [hidden email] https://www.iam.unibe.ch/mailman/listinfo/moose-dev |
Re: Moose 5.1 image getting really sluggish after graphing a bigger dataset
|
|
Hi Sven,
On Mon, Mar 16, 2015 at 7:14 PM, Sven Van Caekenberghe <[hidden email]> wrote: Hi Manfred, thanks for your input. This will come in handy once the files get bigger. But the problem is not in reading the file. Even when I use 'reader upToEnd' it is really fast and all following actions remain fluent. The problem I am facing is that any action I am performing _after_ drawing the graph takes several seconds. And I just noticed that switching from the Canvas to the Raw tab of the TRCanvas object makes the problem disappear. But switching to the Canvas tab itself takes several seconds (probably until the graph is rendered) and every action afterwards is getting delayed several seconds. The machine I am running this image on has 8 GB RAM and a 2.8 GHz Core i7 (mobile) processor if this information helps. Best, Manfred HTH, _______________________________________________ Moose-dev mailing list [hidden email] https://www.iam.unibe.ch/mailman/listinfo/moose-dev |
|
|
Hi Manfred,
You may have pin-pointing a limitation in Roassal. However, I am sure we can handle it easily. Can I have this CSV file? to reproduce the situation on my machine? Resend the complete script as well please :-) Cheers, Alexandre
--
_,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: Alexandre Bergel http://www.bergel.eu ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.
_______________________________________________ Moose-dev mailing list [hidden email] https://www.iam.unibe.ch/mailman/listinfo/moose-dev |
Re: Moose 5.1 image getting really sluggish after graphing a bigger dataset
|
|
Also, does the responsiveness get better after you close the visualization? Cheers, Doru On Mon, Mar 16, 2015 at 9:03 PM, Alexandre Bergel <[hidden email]> wrote:
_______________________________________________ Moose-dev mailing list [hidden email] https://www.iam.unibe.ch/mailman/listinfo/moose-dev |
Re: Moose 5.1 image getting really sluggish after graphing a bigger dataset
|
|
How much RAM is this using? Phil Le 17 mars 2015 07:03, "Tudor Girba" <[hidden email]> a écrit :
_______________________________________________ Moose-dev mailing list [hidden email] https://www.iam.unibe.ch/mailman/listinfo/moose-dev |
Re: Moose 5.1 image getting really sluggish after graphing a bigger dataset
|
|
How much RAM is what using? Doru On Tue, Mar 17, 2015 at 7:09 AM, [hidden email] <[hidden email]> wrote:
_______________________________________________ Moose-dev mailing list [hidden email] https://www.iam.unibe.ch/mailman/listinfo/moose-dev |
Re: Moose 5.1 image getting really sluggish after graphing a bigger dataset
|
|
Thanks for the response so far. Here is a link to the file I am using with the script from my first post. The only thing which needs to be done is to create a Filesystem object for the file with cpuFile := 'path/to/cpu.trace' asFileReference cpuReader := NeoCSVReader on: (cpuFile readStream). cpuReader separator: $|. cpuHeader := cpuReader readHeader. cpuData := cpuReader upToEnd. g := RTGrapher new. g extent: 300@200. ds := RTDataSet new. ds points: (cpuData collect: [ :each | | timestamp cpuUsage | timestamp := (each at: 1) asNumber. cpuUsage := (each at: 4) asNumber. Array braceWith: timestamp with: cpuUsage]). ds x: #first. ds y: #second. ds connectColor: Color blue. g add: ds. g addDecorator: RTHorizontalTickLineDecorator new. g addDecorator: RTVerticalTickLineDecorator new. g build. g view canvas. The responsiveness does get back to normal when I switch from the Canvas view to the Raw or Meta view of the TRCanvas object but also when the tab of the TRCanvas object gets closed. To measure the memory usage of the vm I used the following statement: ((Smalltalk vm getParameters at: 3) / 1000000) asInteger This is adapted from the following SO answer: http://stackoverflow.com/questions/8556207/in-pharo-how-can-you-measure-the-systems-current-total-memory-consumption Before executing the script the statement shows 76 and afterwards 203. Wrapping the whole script in 'MessageTally spyOn: []' gives me the following report: Thanks for your help so far, Manfred On Tue, Mar 17, 2015 at 7:32 AM, Tudor Girba <[hidden email]> wrote:
_______________________________________________ Moose-dev mailing list [hidden email] https://www.iam.unibe.ch/mailman/listinfo/moose-dev |
|
|
Hi Manfred,
Can I have access to the cpu.trace file? I promise to not share it :-) Alexandre
--
_,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: Alexandre Bergel http://www.bergel.eu ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.
_______________________________________________ Moose-dev mailing list [hidden email] https://www.iam.unibe.ch/mailman/listinfo/moose-dev |
|
|
In reply to this post by Manfred Kröhnert
Hi Manfred,
Can I have access to the cpu.trace file? I promise to not share it :-) Alexandre
--
_,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: Alexandre Bergel http://www.bergel.eu ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.
_______________________________________________ Moose-dev mailing list [hidden email] https://www.iam.unibe.ch/mailman/listinfo/moose-dev |
Re: Moose 5.1 image getting really sluggish after graphing a bigger dataset
|
|
In reply to this post by abergel
Hi Alexandre, seems like I forgot to add the link. So here it is: Thanks for taking a look, Manfred On Tue, Mar 17, 2015 at 3:24 PM, Alexandre Bergel <[hidden email]> wrote:
_______________________________________________ Moose-dev mailing list [hidden email] https://www.iam.unibe.ch/mailman/listinfo/moose-dev |
|
|
Hi Manfred,
I think have fixed your problem. Consider this script: -=-=-=-=-=-=-=-=-=-=-=-= cpuFile := '/Users/alexandrebergel/Downloads/cpu.trace' asFileReference. cpuReader := NeoCSVReader on: (cpuFile readStream). cpuReader separator: $|. cpuHeader := cpuReader readHeader. cpuData := cpuReader upToEnd. g := RTGrapher new. g extent: 300@200. ds := RTDataSet new. ds points: (cpuData collect: [ :each | | timestamp cpuUsage | timestamp := (each at: 1) asNumber. cpuUsage := (each at: 4) asNumber. Array braceWith: timestamp with: cpuUsage]). ds x: #first. ds y: #second. ds connectColor: Color blue. g add: ds. g addDecorator: RTHorizontalTickLineDecorator new. g addDecorator: RTVerticalTickLineDecorator new. g build. RTCache new cacheView: g view. g view. -=-=-=-=-=-=-=-=-=-=-=-= The magic line here is "RTCache new cacheView: g view.” I will describe this in another thread. Cheers, Alexandre > On Mar 17, 2015, at 1:47 PM, Manfred Kröhnert <[hidden email]> wrote: > > Hi Alexandre, > > seems like I forgot to add the link. > So here it is: > > https://drive.google.com/file/d/0B-EyIKkOzbhybjdyU2dsa29EODA/view?usp=sharing > > Thanks for taking a look, > Manfred > > > > On Tue, Mar 17, 2015 at 3:24 PM, Alexandre Bergel <[hidden email]> wrote: > Hi Manfred, > > Can I have access to the cpu.trace file? I promise to not share it :-) > > Alexandre > -- > _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: > Alexandre Bergel http://www.bergel.eu > ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;. > > > >> On Mar 17, 2015, at 7:14 AM, Manfred Kröhnert <[hidden email]> wrote: >> >> Thanks for the response so far. >> >> Here is a link to the file I am using with the script from my first post. >> The only thing which needs to be done is to create a Filesystem object for the file with >> >> >> cpuFile := 'path/to/cpu.trace' asFileReference >> cpuReader := NeoCSVReader on: (cpuFile readStream). >> cpuReader separator: $|. >> cpuHeader := cpuReader readHeader. >> cpuData := cpuReader upToEnd. >> >> g := RTGrapher new. >> g extent: 300@200. >> >> ds := RTDataSet new. >> ds points: (cpuData collect: [ :each | >> | timestamp cpuUsage | >> timestamp := (each at: 1) asNumber. >> cpuUsage := (each at: 4) asNumber. >> Array braceWith: timestamp with: cpuUsage]). >> ds x: #first. >> ds y: #second. >> ds connectColor: Color blue. >> >> g add: ds. >> g addDecorator: RTHorizontalTickLineDecorator new. >> g addDecorator: RTVerticalTickLineDecorator new. >> g build. >> g view canvas. >> >> >> The responsiveness does get back to normal when I switch from the Canvas view to the Raw or Meta view of the TRCanvas object but also when the tab of the TRCanvas object gets closed. >> >> To measure the memory usage of the vm I used the following statement: ((Smalltalk vm getParameters at: 3) / 1000000) asInteger >> This is adapted from the following SO answer: http://stackoverflow.com/questions/8556207/in-pharo-how-can-you-measure-the-systems-current-total-memory-consumption >> Before executing the script the statement shows 76 and afterwards 203. >> >> Wrapping the whole script in 'MessageTally spyOn: []' gives me the following report: >> https://drive.google.com/file/d/0B-EyIKkOzbhydU9PNzVjMmZ5d3c/view?usp=sharing >> >> Thanks for your help so far, >> Manfred >> >> >> On Tue, Mar 17, 2015 at 7:32 AM, Tudor Girba <[hidden email]> wrote: >> How much RAM is what using? >> >> Doru >> >> On Tue, Mar 17, 2015 at 7:09 AM, [hidden email] <[hidden email]> wrote: >> How much RAM is this using? >> >> Phil >> >> Le 17 mars 2015 07:03, "Tudor Girba" <[hidden email]> a écrit : >> >> Also, does the responsiveness get better after you close the visualization? >> >> Cheers, >> Doru >> >> On Mon, Mar 16, 2015 at 9:03 PM, Alexandre Bergel <[hidden email]> wrote: >> Hi Manfred, >> >> You may have pin-pointing a limitation in Roassal. However, I am sure we can handle it easily. Can I have this CSV file? to reproduce the situation on my machine? >> >> Resend the complete script as well please :-) >> >> Cheers, >> Alexandre >> -- >> _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: >> Alexandre Bergel http://www.bergel.eu >> ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;. >> >> >> >>> On Mar 16, 2015, at 3:31 PM, Manfred Kröhnert <[hidden email]> wrote: >>> >>> Hi Sven, >>> >>> >>> >>> On Mon, Mar 16, 2015 at 7:14 PM, Sven Van Caekenberghe <[hidden email]> wrote: >>> Hi Manfred, >>> >>> I can only help you with the NeoCSV & IO part. Here is how I would do it: >>> >>> [ >>> 'manfred.csv' asFileReference writeStreamDo: [ :out | >>> (NeoCSVWriter on: (ZnBufferedWriteStream on: out)) in: [ :writer | >>> writer separator: $|; writeHeader: #(timestamp two three cpu); fieldWriter: #raw. >>> 130000 timesRepeat: [ >>> writer nextPut: { DateAndTime now asUnixTime. 999 atRandom. 999 atRandom. 999 atRandom } ] ] ]. >>> ] timeToRun. "0:00:00:01.917" >>> >>> [ >>> 'manfred.csv' asFileReference readStreamDo: [ :in | >>> (NeoCSVReader on: (ZnBufferedReadStream on: in)) in: [ :reader | >>> reader separator: $|; readHeader. >>> reader addIntegerField; addIgnoredField; addIgnoredField; addIntegerField. >>> reader upToEnd ] ]. >>> ] timeToRun. "0:00:00:01.064" >>> >>> I think that is acceptable for a 3Mb file. >>> >>> >>> thanks for your input. >>> This will come in handy once the files get bigger. >>> >>> But the problem is not in reading the file. >>> Even when I use 'reader upToEnd' it is really fast and all following actions remain fluent. >>> >>> The problem I am facing is that any action I am performing _after_ drawing the graph takes several seconds. >>> And I just noticed that switching from the Canvas to the Raw tab of the TRCanvas object makes the problem disappear. >>> But switching to the Canvas tab itself takes several seconds (probably until the graph is rendered) and every action afterwards is getting delayed several seconds. >>> >>> The machine I am running this image on has 8 GB RAM and a 2.8 GHz Core i7 (mobile) processor if this information helps. >>> >>> Best, >>> Manfred >>> >>> >>> >>> HTH, >>> >>> Sven >>> >>> > On 16 Mar 2015, at 17:48, Manfred Kröhnert <[hidden email]> wrote: >>> > >>> > Hi everyone, >>> > >>> > after having access to a machine running OS X 10.10.2 I started to give Moose 5.1 a try for some initial data analysis. >>> > I used Pharo Launcher (build #24) and downloaded the Moose image with buildnumber 304. >>> > >>> > The task is to read a CSV log file containing around 130000 entries, each containing a timestamp, a cpu usage number, and some other information. >>> > Everything works as expected, but the responsiveness of the image drops dramatically after executing the script in Playground with Cmd+Shift+g. >>> > After evaluating the script it takes several seconds to get a response to any action such as opening the world menu or switching tabs in the playground. >>> > >>> > Is this because I am doing something very inefficient (most probably) or does this problem occur more often? >>> > Here is the script I am using: >>> > >>> > >>> > cpuReader := NeoCSVReader on: (cpuFile readStream). >>> > cpuReader separator: $|. >>> > cpuHeader := cpuReader readHeader. >>> > cpuData := cpuReader upToEnd. >>> > >>> > g := RTGrapher new. >>> > g extent: 300@200. >>> > >>> > ds := RTDataSet new. >>> > ds points: (cpuData collect: [ :each | >>> > | timestamp cpuUsage | >>> > timestamp := (each at: 1) asNumber. >>> > cpuUsage := (each at: 4) asNumber. >>> > Array braceWith: timestamp with: cpuUsage]). >>> > ds x: #first. >>> > ds y: #second. >>> > ds connectColor: Color blue. >>> > >>> > g add: ds. >>> > g addDecorator: RTHorizontalTickLineDecorator new. >>> > g addDecorator: RTVerticalTickLineDecorator new. >>> > g build. >>> > g view canvas. >>> > >>> > >>> > I also noticed that the 'Meta' representation of an object looks very crammed since it does not use the full size available in the tab. >>> > Should issues like this be reported somewhere else? >>> > >>> > Thanks for taking a look at this, >>> > Manfred >>> > _______________________________________________ >>> > Moose-dev mailing list >>> > [hidden email] >>> > https://www.iam.unibe.ch/mailman/listinfo/moose-dev >>> >>> >>> _______________________________________________ >>> Moose-dev mailing list >>> [hidden email] >>> https://www.iam.unibe.ch/mailman/listinfo/moose-dev >>> >>> _______________________________________________ >>> Moose-dev mailing list >>> [hidden email] >>> https://www.iam.unibe.ch/mailman/listinfo/moose-dev >> >> >> _______________________________________________ >> Moose-dev mailing list >> [hidden email] >> https://www.iam.unibe.ch/mailman/listinfo/moose-dev >> >> >> >> >> -- >> www.tudorgirba.com >> >> "Every thing has its own flow" >> >> _______________________________________________ >> Moose-dev mailing list >> [hidden email] >> https://www.iam.unibe.ch/mailman/listinfo/moose-dev >> >> >> _______________________________________________ >> Moose-dev mailing list >> [hidden email] >> https://www.iam.unibe.ch/mailman/listinfo/moose-dev >> >> >> >> >> -- >> www.tudorgirba.com >> >> "Every thing has its own flow" >> >> _______________________________________________ >> Moose-dev mailing list >> [hidden email] >> https://www.iam.unibe.ch/mailman/listinfo/moose-dev >> >> >> _______________________________________________ >> Moose-dev mailing list >> [hidden email] >> https://www.iam.unibe.ch/mailman/listinfo/moose-dev > > > _______________________________________________ > Moose-dev mailing list > [hidden email] > https://www.iam.unibe.ch/mailman/listinfo/moose-dev > > > _______________________________________________ > Moose-dev mailing list > [hidden email] > https://www.iam.unibe.ch/mailman/listinfo/moose-dev -- _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: Alexandre Bergel http://www.bergel.eu ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;. _______________________________________________ Moose-dev mailing list [hidden email] https://www.iam.unibe.ch/mailman/listinfo/moose-dev |
Re: Moose 5.1 image getting really sluggish after graphing a bigger dataset
|
|
Thanks Alex, adding the RTCache line to the script helps preventing the unresponsiveness in the most recent Moose image. It's only that this feels more like a workaround since this is not remotely obvious for a beginner like me :-) Would it be possible to let the preview tab in the inspector do something like this by default? Then it would work out of the box. Thanks again, Manfred On Wed, Mar 18, 2015 at 12:45 AM, Alexandre Bergel <[hidden email]> wrote: Hi Manfred, _______________________________________________ Moose-dev mailing list [hidden email] https://www.iam.unibe.ch/mailman/listinfo/moose-dev |
|
|
Hi!
Making things scalable is indeed a pain for the user. Currently, using RTCache remove all the interactions from the elements. This means we cannot have this cache all the time. At least for now. Also, it may create oddities with labels. You have a large data set to render, which is not the case of everyone. Is the current solution acceptable for now? Cheers, Alexandre
--
_,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: Alexandre Bergel http://www.bergel.eu ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.
_______________________________________________ Moose-dev mailing list [hidden email] https://www.iam.unibe.ch/mailman/listinfo/moose-dev |
Re: Moose 5.1 image getting really sluggish after graphing a bigger dataset
|
|
Hi Alexandre,
On Wed, Mar 18, 2015 at 2:01 PM, Alexandre Bergel <[hidden email]> wrote:
thanks for the explanation. In my case I don't need interactive features (at least for now).
I can work with the current solution, now that I know of the cache. Thanks, Manfred
_______________________________________________ Moose-dev mailing list [hidden email] https://www.iam.unibe.ch/mailman/listinfo/moose-dev |
|
|
Ok, let us know. This is important for us that you feel confortable with Roassal
Alexandre
--
_,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: Alexandre Bergel http://www.bergel.eu ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.
_______________________________________________ Moose-dev mailing list [hidden email] https://www.iam.unibe.ch/mailman/listinfo/moose-dev |
«
Return to Moose
|
1 view|%1 views
| Free forum by Nabble | Edit this page |