Raw pane on byteStrings

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
21 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Raw pane on byteStrings

stepharong
Hi

I'm trying to debug citezen generation and I have to compare strings.
Now I think that the raw views (in Pharo 50) is not good because we cannot  
see all the items in raw format.
See the attachements. It jumps from 21 to 174 ...
and what I want to see is of course in the middle.

Is it me or there is something wrong there.
Stef

Screen Shot 2017-02-23 at 19.32.21.png (95K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Raw pane on byteStrings

Andrei Chis
Hi Stef,

Currently that's the default behaviour of the Raw view: it displays for collections only the first and the last 21 elements. The Items view however always should display all the elements of a collection.

The main problem with the Raw view in Pharo 5 is the speed. In Pharo 6 now the speed of the Raw view is greately improved so we could increase those limits. Still for now there should still be some limit for the Raw view. Ideally we should add a small widget, something like a paginator, for navigating through large and very large collections.

Cheers,
Andrei

On Feb 23, 2017 19:35, "stepharong" <[hidden email]> wrote:
Hi

I'm trying to debug citezen generation and I have to compare strings.
Now I think that the raw views (in Pharo 50) is not good because we cannot see all the items in raw format.
See the attachements. It jumps from 21 to 174 ...
and what I want to see is of course in the middle.

Is it me or there is something wrong there.
Stef
Reply | Threaded
Open this post in threaded view
|

Re: Raw pane on byteStrings

stepharong
Hi andrei

Yes. The problem is that it makes the raw view sometimes totally useless as in my case. 
I could not see anything I was interesting in. 
I found the pagniator confusing a bit because it expands in the middle (it means that when I want to see 54 and over then I have to scroll in the middle
because 50 is not a the top like in a book page. 

Stef

Hi Stef,

Currently that's the default behaviour of the Raw view: it displays for collections only the first and the last 21 elements. The Items view however always should display all the elements of a collection.

The main problem with the Raw view in Pharo 5 is the speed. In Pharo 6 now the speed of the Raw view is greately improved so we could increase those limits. Still for now there should still be some limit for the Raw view. Ideally we should add a small widget, something like a paginator, for navigating through large and very large collections.

Cheers,
Andrei

On Feb 23, 2017 19:35, "stepharong" <[hidden email]> wrote:
Hi

I'm trying to debug citezen generation and I have to compare strings.
Now I think that the raw views (in Pharo 50) is not good because we cannot see all the items in raw format.
See the attachements. It jumps from 21 to 174 ...
and what I want to see is of course in the middle.

Is it me or there is something wrong there.
Stef



--
Using Opera's mail client: http://www.opera.com/mail/
Reply | Threaded
Open this post in threaded view
|

Re: Raw pane on byteStrings

Ben Coman
In reply to this post by Andrei Chis
A couple of random side-thoughts for Pharo 7. Would it be possible...
* to have a "diff" tab, for example to compare two strings side by side, perhaps highlighting and being able to jump to the differences.
* to somehow reference the parent pane, for example if the current pane shows a string element from an array, I can grab another string from the same array to diff against
* coming at it from the other way, perhaps an array [Items] tab could be multi-selected for elements to diff 

* it would be nice if you could right-click on a tab to get a menu item to open its implementation. Then in this case Stef might jump to the implementation of the [Raw], change the tab's name, remove the code skipping middle elements and in a few minutes have a new tab doing what he wants.  
* perhaps the tab menu item could be called "Customize" and it automatically copies and renames the [Raw] tab implementation into the String class so its ready for editing.

cheers -ben

On Fri, Feb 24, 2017 at 3:19 AM, Andrei Chis <[hidden email]> wrote:
Hi Stef,

Currently that's the default behaviour of the Raw view: it displays for collections only the first and the last 21 elements. The Items view however always should display all the elements of a collection.

The main problem with the Raw view in Pharo 5 is the speed. In Pharo 6 now the speed of the Raw view is greately improved so we could increase those limits. Still for now there should still be some limit for the Raw view. Ideally we should add a small widget, something like a paginator, for navigating through large and very large collections.

Cheers,
Andrei

On Feb 23, 2017 19:35, "stepharong" <[hidden email]> wrote:
Hi

I'm trying to debug citezen generation and I have to compare strings.
Now I think that the raw views (in Pharo 50) is not good because we cannot see all the items in raw format.
See the attachements. It jumps from 21 to 174 ...
and what I want to see is of course in the middle.

Is it me or there is something wrong there.
Stef

Reply | Threaded
Open this post in threaded view
|

Re: Raw pane on byteStrings

Denis Kudriashov
In reply to this post by Andrei Chis

2017-02-23 20:19 GMT+01:00 Andrei Chis <[hidden email]>:
Hi Stef,

Currently that's the default behaviour of the Raw view: it displays for collections only the first and the last 21 elements. The Items view however always should display all the elements of a collection.

The main problem with the Raw view in Pharo 5 is the speed. In Pharo 6 now the speed of the Raw view is greately improved so we could increase those limits. Still for now there should still be some limit for the Raw view. Ideally we should add a small widget, something like a paginator, for navigating through large and very large collections.

I wondering, does not it fixed with fast table integration?
Reply | Threaded
Open this post in threaded view
|

Re: Raw pane on byteStrings

Thierry Goubier
In reply to this post by Andrei Chis
Hi Andrei,

if you're using fasttable for the Raw view, you should be able to reach one 100k elements without issues. I did some experiments and it handles the load very well.

Avoid the paginator at any cost. This thing is really user-unfriendly.

Regards,

Thierry

2017-02-23 20:19 GMT+01:00 Andrei Chis <[hidden email]>:
Hi Stef,

Currently that's the default behaviour of the Raw view: it displays for collections only the first and the last 21 elements. The Items view however always should display all the elements of a collection.

The main problem with the Raw view in Pharo 5 is the speed. In Pharo 6 now the speed of the Raw view is greately improved so we could increase those limits. Still for now there should still be some limit for the Raw view. Ideally we should add a small widget, something like a paginator, for navigating through large and very large collections.

Cheers,
Andrei

On Feb 23, 2017 19:35, "stepharong" <[hidden email]> wrote:
Hi

I'm trying to debug citezen generation and I have to compare strings.
Now I think that the raw views (in Pharo 50) is not good because we cannot see all the items in raw format.
See the attachements. It jumps from 21 to 174 ...
and what I want to see is of course in the middle.

Is it me or there is something wrong there.
Stef

Reply | Threaded
Open this post in threaded view
|

Re: Raw pane on byteStrings

Denis Kudriashov

2017-02-24 10:37 GMT+01:00 Thierry Goubier <[hidden email]>:
Hi Andrei,

if you're using fasttable for the Raw view, you should be able to reach one 100k elements without issues. I did some experiments and it handles the load very well.

Avoid the paginator at any cost. This thing is really user-unfriendly.

+100
Reply | Threaded
Open this post in threaded view
|

Re: Raw pane on byteStrings

Andrei Chis
In reply to this post by Thierry Goubier
Hi Thierry,

Indeed that's the simplest option now that we are using fast table.

Just now in the case of the Raw view an OrderedCollection is used to store all displayed elements.
If you display large collections every time you open the Raw view it will instantiate a collection of size 100k and instantiate 100k objects of type GTInspectorIndexedNode. With FastTable we can lazily load elements so we should be able to remove this behaviour and the limit. Just we need to make sure it will play nicely with automatic refresh. There is also the issue that when expanding an element in the tree if it's a collection you don't want to expand all elements.

I'm looking now on a lazy data source for FastTable that plays nicely with GTInspector. Let's see how it will go. Help is always welcomed :)

I think I used the word paginator in the wrong way. If you have a very large collection (millions of elements) I do not want to scroll through elements but most likely jump to a certain element or view just a subset of all elements. Not really add a paginator like in web pages.

Cheers,
Andrei

On Fri, Feb 24, 2017 at 10:37 AM, Thierry Goubier <[hidden email]> wrote:
Hi Andrei,

if you're using fasttable for the Raw view, you should be able to reach one 100k elements without issues. I did some experiments and it handles the load very well.

Avoid the paginator at any cost. This thing is really user-unfriendly.

Regards,

Thierry

2017-02-23 20:19 GMT+01:00 Andrei Chis <[hidden email]>:
Hi Stef,

Currently that's the default behaviour of the Raw view: it displays for collections only the first and the last 21 elements. The Items view however always should display all the elements of a collection.

The main problem with the Raw view in Pharo 5 is the speed. In Pharo 6 now the speed of the Raw view is greately improved so we could increase those limits. Still for now there should still be some limit for the Raw view. Ideally we should add a small widget, something like a paginator, for navigating through large and very large collections.

Cheers,
Andrei

On Feb 23, 2017 19:35, "stepharong" <[hidden email]> wrote:
Hi

I'm trying to debug citezen generation and I have to compare strings.
Now I think that the raw views (in Pharo 50) is not good because we cannot see all the items in raw format.
See the attachements. It jumps from 21 to 174 ...
and what I want to see is of course in the middle.

Is it me or there is something wrong there.
Stef


Reply | Threaded
Open this post in threaded view
|

Re: Raw pane on byteStrings

Thierry Goubier
Hi Andrei,

2017-02-24 11:31 GMT+01:00 Andrei Chis <[hidden email]>:
Hi Thierry,

Indeed that's the simplest option now that we are using fast table.

Just now in the case of the Raw view an OrderedCollection is used to store all displayed elements.
If you display large collections every time you open the Raw view it will instantiate a collection of size 100k and instantiate 100k objects of type GTInspectorIndexedNode. With FastTable we can lazily load elements so we should be able to remove this behaviour and the limit. Just we need to make sure it will play nicely with automatic refresh. There is also the issue that when expanding an element in the tree if it's a collection you don't want to expand all elements.

I'm not sure you need to worry too much about that one; in practical experiments, creating that 100k collection for viewing (and the associated nodes instances) isn't too costly (unless creating the GTInspectorIndexedNodes has hidden costs: I've only experimented with the EyeInspector framework).

Opening the tree with all elements works fine in my experiments. Tuning scrolling as done in Bloc is necessary, however. 
 

I'm looking now on a lazy data source for FastTable that plays nicely with GTInspector. Let's see how it will go. Help is always welcomed :)

As I said: do not overoptimize that part... just remove that limitation on the raw view and measure.
 

I think I used the word paginator in the wrong way. If you have a very large collection (millions of elements) I do not want to scroll through elements but most likely jump to a certain element or view just a subset of all elements. Not really add a paginator like in web pages.

Ok, millions of elements is a bit out of my scope... I'll look for filtering then.

Regards,

Thierry
 

Cheers,
Andrei

On Fri, Feb 24, 2017 at 10:37 AM, Thierry Goubier <[hidden email]> wrote:
Hi Andrei,

if you're using fasttable for the Raw view, you should be able to reach one 100k elements without issues. I did some experiments and it handles the load very well.

Avoid the paginator at any cost. This thing is really user-unfriendly.

Regards,

Thierry

2017-02-23 20:19 GMT+01:00 Andrei Chis <[hidden email]>:
Hi Stef,

Currently that's the default behaviour of the Raw view: it displays for collections only the first and the last 21 elements. The Items view however always should display all the elements of a collection.

The main problem with the Raw view in Pharo 5 is the speed. In Pharo 6 now the speed of the Raw view is greately improved so we could increase those limits. Still for now there should still be some limit for the Raw view. Ideally we should add a small widget, something like a paginator, for navigating through large and very large collections.

Cheers,
Andrei

On Feb 23, 2017 19:35, "stepharong" <[hidden email]> wrote:
Hi

I'm trying to debug citezen generation and I have to compare strings.
Now I think that the raw views (in Pharo 50) is not good because we cannot see all the items in raw format.
See the attachements. It jumps from 21 to 174 ...
and what I want to see is of course in the middle.

Is it me or there is something wrong there.
Stef



Reply | Threaded
Open this post in threaded view
|

Re: Raw pane on byteStrings

Andrei Chis


On Fri, Feb 24, 2017 at 12:16 PM, Thierry Goubier <[hidden email]> wrote:
Hi Andrei,

2017-02-24 11:31 GMT+01:00 Andrei Chis <[hidden email]>:
Hi Thierry,

Indeed that's the simplest option now that we are using fast table.

Just now in the case of the Raw view an OrderedCollection is used to store all displayed elements.
If you display large collections every time you open the Raw view it will instantiate a collection of size 100k and instantiate 100k objects of type GTInspectorIndexedNode. With FastTable we can lazily load elements so we should be able to remove this behaviour and the limit. Just we need to make sure it will play nicely with automatic refresh. There is also the issue that when expanding an element in the tree if it's a collection you don't want to expand all elements.

I'm not sure you need to worry too much about that one; in practical experiments, creating that 100k collection for viewing (and the associated nodes instances) isn't too costly (unless creating the GTInspectorIndexedNodes has hidden costs: I've only experimented with the EyeInspector framework).

There should be no hidden costs in GTInspectorIndexedNodes.
I made some experiments in the latest Pharo version and opening the Raw view on an array with one million numbers takes around 120ms when 100k elements are computed.
I'll be curious how much it takes on your machine. To test update indexableDisplayLimit to 50000 in Object>>#gtInspectorVariableNodesIn: and remove the annotation from Collection>>#gtInspectorItemsIn: (so that the Items presentation is not loaded)

arrayLarge := (1 to: 1000000) asArray.
arrayLarge inspect.
 

Opening the tree with all elements works fine in my experiments. Tuning scrolling as done in Bloc is necessary, however. 
 

I'm looking now on a lazy data source for FastTable that plays nicely with GTInspector. Let's see how it will go. Help is always welcomed :)

As I said: do not overoptimize that part... just remove that limitation on the raw view and measure.

If I measure the Items view on the previous array it takes around 35ms.
What I'd like to have is the same opening time for the Items view on large Sets and Dictionaries.
 
 

I think I used the word paginator in the wrong way. If you have a very large collection (millions of elements) I do not want to scroll through elements but most likely jump to a certain element or view just a subset of all elements. Not really add a paginator like in web pages.

Ok, millions of elements is a bit out of my scope... I'll look for filtering then.

Yes, definitely filtering is the way to go there :)

Cheers,
Andrei
 

Regards,

Thierry
 

Cheers,
Andrei

On Fri, Feb 24, 2017 at 10:37 AM, Thierry Goubier <[hidden email]> wrote:
Hi Andrei,

if you're using fasttable for the Raw view, you should be able to reach one 100k elements without issues. I did some experiments and it handles the load very well.

Avoid the paginator at any cost. This thing is really user-unfriendly.

Regards,

Thierry

2017-02-23 20:19 GMT+01:00 Andrei Chis <[hidden email]>:
Hi Stef,

Currently that's the default behaviour of the Raw view: it displays for collections only the first and the last 21 elements. The Items view however always should display all the elements of a collection.

The main problem with the Raw view in Pharo 5 is the speed. In Pharo 6 now the speed of the Raw view is greately improved so we could increase those limits. Still for now there should still be some limit for the Raw view. Ideally we should add a small widget, something like a paginator, for navigating through large and very large collections.

Cheers,
Andrei

On Feb 23, 2017 19:35, "stepharong" <[hidden email]> wrote:
Hi

I'm trying to debug citezen generation and I have to compare strings.
Now I think that the raw views (in Pharo 50) is not good because we cannot see all the items in raw format.
See the attachements. It jumps from 21 to 174 ...
and what I want to see is of course in the middle.

Is it me or there is something wrong there.
Stef




Reply | Threaded
Open this post in threaded view
|

Re: Raw pane on byteStrings

Thierry Goubier


2017-02-24 14:29 GMT+01:00 Andrei Chis <[hidden email]>:


On Fri, Feb 24, 2017 at 12:16 PM, Thierry Goubier <[hidden email]> wrote:
Hi Andrei,

2017-02-24 11:31 GMT+01:00 Andrei Chis <[hidden email]>:
Hi Thierry,

Indeed that's the simplest option now that we are using fast table.

Just now in the case of the Raw view an OrderedCollection is used to store all displayed elements.
If you display large collections every time you open the Raw view it will instantiate a collection of size 100k and instantiate 100k objects of type GTInspectorIndexedNode. With FastTable we can lazily load elements so we should be able to remove this behaviour and the limit. Just we need to make sure it will play nicely with automatic refresh. There is also the issue that when expanding an element in the tree if it's a collection you don't want to expand all elements.

I'm not sure you need to worry too much about that one; in practical experiments, creating that 100k collection for viewing (and the associated nodes instances) isn't too costly (unless creating the GTInspectorIndexedNodes has hidden costs: I've only experimented with the EyeInspector framework).

There should be no hidden costs in GTInspectorIndexedNodes.
I made some experiments in the latest Pharo version and opening the Raw view on an array with one million numbers takes around 120ms when 100k elements are computed.
I'll be curious how much it takes on your machine. To test update indexableDisplayLimit to 50000 in Object>>#gtInspectorVariableNodesIn: and remove the annotation from Collection>>#gtInspectorItemsIn: (so that the Items presentation is not loaded)

arrayLarge := (1 to: 1000000) asArray.
arrayLarge inspect.

This is the values I get on my work laptop (core i3-2350M 2.30 Ghz); both inspectors displays 100k elements.

(1 to: 1000000) asArray in: [ :s |  [s inspect] timeToRun] 0:00:00:00.064
(1 to: 1000000) asArray in: [:s | [GTInspector inspect: s] timeToRun] 0:00:00:00.381

Pharo 6 60411
 
 

Opening the tree with all elements works fine in my experiments. Tuning scrolling as done in Bloc is necessary, however. 
 

I'm looking now on a lazy data source for FastTable that plays nicely with GTInspector. Let's see how it will go. Help is always welcomed :)

As I said: do not overoptimize that part... just remove that limitation on the raw view and measure.

If I measure the Items view on the previous array it takes around 35ms.
What I'd like to have is the same opening time for the Items view on large Sets and Dictionaries.

On my machine, the experiment is that displaying the set is fast, but the system becomes totally unresponsive... which may be an issue with the self refresh of the inspector. Yes, it was the culprit.

(1 to: 1000000) asSet in: [ :s |  [s inspect] timeToRun] 0:00:00:00.034
(1 to: 1000000) asSet in: [:s | [GTInspector inspect: s] timeToRun] 0:00:00:00.199

Regards,

Thierry
 
 
 

I think I used the word paginator in the wrong way. If you have a very large collection (millions of elements) I do not want to scroll through elements but most likely jump to a certain element or view just a subset of all elements. Not really add a paginator like in web pages.

Ok, millions of elements is a bit out of my scope... I'll look for filtering then.

Yes, definitely filtering is the way to go there :)

Cheers,
Andrei
 

Regards,

Thierry
 

Cheers,
Andrei

On Fri, Feb 24, 2017 at 10:37 AM, Thierry Goubier <[hidden email]> wrote:
Hi Andrei,

if you're using fasttable for the Raw view, you should be able to reach one 100k elements without issues. I did some experiments and it handles the load very well.

Avoid the paginator at any cost. This thing is really user-unfriendly.

Regards,

Thierry

2017-02-23 20:19 GMT+01:00 Andrei Chis <[hidden email]>:
Hi Stef,

Currently that's the default behaviour of the Raw view: it displays for collections only the first and the last 21 elements. The Items view however always should display all the elements of a collection.

The main problem with the Raw view in Pharo 5 is the speed. In Pharo 6 now the speed of the Raw view is greately improved so we could increase those limits. Still for now there should still be some limit for the Raw view. Ideally we should add a small widget, something like a paginator, for navigating through large and very large collections.

Cheers,
Andrei

On Feb 23, 2017 19:35, "stepharong" <[hidden email]> wrote:
Hi

I'm trying to debug citezen generation and I have to compare strings.
Now I think that the raw views (in Pharo 50) is not good because we cannot see all the items in raw format.
See the attachements. It jumps from 21 to 174 ...
and what I want to see is of course in the middle.

Is it me or there is something wrong there.
Stef





Reply | Threaded
Open this post in threaded view
|

Re: Raw pane on byteStrings

Denis Kudriashov
In reply to this post by Andrei Chis

2017-02-24 14:29 GMT+01:00 Andrei Chis <[hidden email]>:
There should be no hidden costs in GTInspectorIndexedNodes.
I made some experiments in the latest Pharo version and opening the Raw view on an array with one million numbers takes around 120ms when 100k elements are computed.
I'll be curious how much it takes on your machine. To test update indexableDisplayLimit to 50000 in Object>>#gtInspectorVariableNodesIn: and remove the annotation from Collection>>#gtInspectorItemsIn: (so that the Items presentation is not loaded)

arrayLarge := (1 to: 1000000) asArray.

I really wondering why anybody want instantiate wrapper objects for all array items? Fast table approach is to not do that. Only visible part of items should be recreated
Reply | Threaded
Open this post in threaded view
|

Re: Raw pane on byteStrings

Andrei Chis


On Fri, Feb 24, 2017 at 3:28 PM, Denis Kudriashov <[hidden email]> wrote:

2017-02-24 14:29 GMT+01:00 Andrei Chis <[hidden email]>:
There should be no hidden costs in GTInspectorIndexedNodes.
I made some experiments in the latest Pharo version and opening the Raw view on an array with one million numbers takes around 120ms when 100k elements are computed.
I'll be curious how much it takes on your machine. To test update indexableDisplayLimit to 50000 in Object>>#gtInspectorVariableNodesIn: and remove the annotation from Collection>>#gtInspectorItemsIn: (so that the Items presentation is not loaded)

arrayLarge := (1 to: 1000000) asArray.

I really wondering why anybody want instantiate wrapper objects for all array items? Fast table approach is to not do that. Only visible part of items should be recreated

This is how the inspector works for the Items view for Array objects. It also does not use any wrappers and only computes visible part.

Reply | Threaded
Open this post in threaded view
|

Re: Raw pane on byteStrings

Thierry Goubier
In reply to this post by Denis Kudriashov


2017-02-24 15:28 GMT+01:00 Denis Kudriashov <[hidden email]>:

2017-02-24 14:29 GMT+01:00 Andrei Chis <[hidden email]>:
There should be no hidden costs in GTInspectorIndexedNodes.
I made some experiments in the latest Pharo version and opening the Raw view on an array with one million numbers takes around 120ms when 100k elements are computed.
I'll be curious how much it takes on your machine. To test update indexableDisplayLimit to 50000 in Object>>#gtInspectorVariableNodesIn: and remove the annotation from Collection>>#gtInspectorItemsIn: (so that the Items presentation is not loaded)

arrayLarge := (1 to: 1000000) asArray.

I really wondering why anybody want instantiate wrapper objects for all array items? Fast table approach is to not do that. Only visible part of items should be recreated

If you want. The fasttable code does the first thing of not creating a wrapper interactive representation for all elements (i.e. a Morph). Then the datasources may also be lazy: only wrap the objects for which a representation is asked. The thing is that to be able to plan that in totally generic UIs like an inspector, is that you need to create placeholders (one way or another) to let the UI knows what kind and how many elements the datasource holds... if the cost is low enough, just creating a placeholder for each element may turn out to be just fine.

And the truth is, in local memory, creating a placeholder for every element in a 100k collection costs about 0.

Remote can be handled either by a lazy source, or just by blindly creating placeholders and caching the accesses to the real remote objects (with the fact that you can expect that remote requests are only done when displaying the element, not when building the widget). Simple, easy, fast enough, and maybe better than considering than a lazy source on a remote source is a ad-hoc implementation of a cache.

Regards,

Thierry

Reply | Threaded
Open this post in threaded view
|

Re: Raw pane on byteStrings

Thierry Goubier
In reply to this post by Andrei Chis


2017-02-24 15:43 GMT+01:00 Andrei Chis <[hidden email]>:


On Fri, Feb 24, 2017 at 3:28 PM, Denis Kudriashov <[hidden email]> wrote:

2017-02-24 14:29 GMT+01:00 Andrei Chis <[hidden email]>:
There should be no hidden costs in GTInspectorIndexedNodes.
I made some experiments in the latest Pharo version and opening the Raw view on an array with one million numbers takes around 120ms when 100k elements are computed.
I'll be curious how much it takes on your machine. To test update indexableDisplayLimit to 50000 in Object>>#gtInspectorVariableNodesIn: and remove the annotation from Collection>>#gtInspectorItemsIn: (so that the Items presentation is not loaded)

arrayLarge := (1 to: 1000000) asArray.

I really wondering why anybody want instantiate wrapper objects for all array items? Fast table approach is to not do that. Only visible part of items should be recreated

This is how the inspector works for the Items view for Array objects. It also does not use any wrappers and only computes visible part.


As I said, it brings in a lazy data source adapted to arrays in a case where it may well be not needed. My experiments show that there is no need for this optimisation, at least for arrays ~ 100k.

Thierry




Reply | Threaded
Open this post in threaded view
|

Re: Raw pane on byteStrings

Andrei Chis
In reply to this post by Thierry Goubier
Hi Thierry,

Strangely enough I'm getting different times on my machine
Just to start from the same baseline in a fresh Pharo 60411 image with no changes to any of the inspectors over a series of runs I get:

array := (1 to: 1000000) asArray.
[array inspect] timeToRun.                       "0:00:00:00.145"
[GTInspector inspect: array] timeToRun. "0:00:00:00.031"

If I change indexableDisplayLimit to 50000 and remove the Items view then I get:

[GTInspector inspect: array] timeToRun.  "0:00:00:00.124"

I'm really interested in seeing what makes GTInspector slower on your machine. Did you do other optimizations to SpecInspector?
I'm using a MacBook Pro Retina - 2.8 GHz Intel Core i7

Pharo VM version:
CoInterpreter VMMaker.oscog-HolgerHansPeterFreyther.1880 uuid: 16138eb3-2390-40f5-a6c8-15f0494936f8 Oct 10 2016
StackToRegisterMappingCogit VMMaker.oscog-HolgerHansPeterFreyther.1880 uuid: 16138eb3-2390-40f5-a6c8-15f0494936f8 Oct 10 2016
[hidden email]:pharo-project/pharo-vm.git Commit: 06744effac0f0aa3b4b32e17636448f9d51d6707 Date: 2016-09-30 08:40:43 +0200 By: GitHub <[hidden email]

Cheers,
Andrei


On Fri, Feb 24, 2017 at 3:24 PM, Thierry Goubier <[hidden email]> wrote:


2017-02-24 14:29 GMT+01:00 Andrei Chis <[hidden email]>:


On Fri, Feb 24, 2017 at 12:16 PM, Thierry Goubier <[hidden email]> wrote:
Hi Andrei,

2017-02-24 11:31 GMT+01:00 Andrei Chis <[hidden email]>:
Hi Thierry,

Indeed that's the simplest option now that we are using fast table.

Just now in the case of the Raw view an OrderedCollection is used to store all displayed elements.
If you display large collections every time you open the Raw view it will instantiate a collection of size 100k and instantiate 100k objects of type GTInspectorIndexedNode. With FastTable we can lazily load elements so we should be able to remove this behaviour and the limit. Just we need to make sure it will play nicely with automatic refresh. There is also the issue that when expanding an element in the tree if it's a collection you don't want to expand all elements.

I'm not sure you need to worry too much about that one; in practical experiments, creating that 100k collection for viewing (and the associated nodes instances) isn't too costly (unless creating the GTInspectorIndexedNodes has hidden costs: I've only experimented with the EyeInspector framework).

There should be no hidden costs in GTInspectorIndexedNodes.
I made some experiments in the latest Pharo version and opening the Raw view on an array with one million numbers takes around 120ms when 100k elements are computed.
I'll be curious how much it takes on your machine. To test update indexableDisplayLimit to 50000 in Object>>#gtInspectorVariableNodesIn: and remove the annotation from Collection>>#gtInspectorItemsIn: (so that the Items presentation is not loaded)

arrayLarge := (1 to: 1000000) asArray.
arrayLarge inspect.

This is the values I get on my work laptop (core i3-2350M 2.30 Ghz); both inspectors displays 100k elements.

(1 to: 1000000) asArray in: [ :s |  [s inspect] timeToRun] 0:00:00:00.064
(1 to: 1000000) asArray in: [:s | [GTInspector inspect: s] timeToRun] 0:00:00:00.381

Pharo 6 60411
 
 

Opening the tree with all elements works fine in my experiments. Tuning scrolling as done in Bloc is necessary, however. 
 

I'm looking now on a lazy data source for FastTable that plays nicely with GTInspector. Let's see how it will go. Help is always welcomed :)

As I said: do not overoptimize that part... just remove that limitation on the raw view and measure.

If I measure the Items view on the previous array it takes around 35ms.
What I'd like to have is the same opening time for the Items view on large Sets and Dictionaries.

On my machine, the experiment is that displaying the set is fast, but the system becomes totally unresponsive... which may be an issue with the self refresh of the inspector. Yes, it was the culprit.

(1 to: 1000000) asSet in: [ :s |  [s inspect] timeToRun] 0:00:00:00.034
(1 to: 1000000) asSet in: [:s | [GTInspector inspect: s] timeToRun] 0:00:00:00.199

Regards,

Thierry
 
 
 

I think I used the word paginator in the wrong way. If you have a very large collection (millions of elements) I do not want to scroll through elements but most likely jump to a certain element or view just a subset of all elements. Not really add a paginator like in web pages.

Ok, millions of elements is a bit out of my scope... I'll look for filtering then.

Yes, definitely filtering is the way to go there :)

Cheers,
Andrei
 

Regards,

Thierry
 

Cheers,
Andrei

On Fri, Feb 24, 2017 at 10:37 AM, Thierry Goubier <[hidden email]> wrote:
Hi Andrei,

if you're using fasttable for the Raw view, you should be able to reach one 100k elements without issues. I did some experiments and it handles the load very well.

Avoid the paginator at any cost. This thing is really user-unfriendly.

Regards,

Thierry

2017-02-23 20:19 GMT+01:00 Andrei Chis <[hidden email]>:
Hi Stef,

Currently that's the default behaviour of the Raw view: it displays for collections only the first and the last 21 elements. The Items view however always should display all the elements of a collection.

The main problem with the Raw view in Pharo 5 is the speed. In Pharo 6 now the speed of the Raw view is greately improved so we could increase those limits. Still for now there should still be some limit for the Raw view. Ideally we should add a small widget, something like a paginator, for navigating through large and very large collections.

Cheers,
Andrei

On Feb 23, 2017 19:35, "stepharong" <[hidden email]> wrote:
Hi

I'm trying to debug citezen generation and I have to compare strings.
Now I think that the raw views (in Pharo 50) is not good because we cannot see all the items in raw format.
See the attachements. It jumps from 21 to 174 ...
and what I want to see is of course in the middle.

Is it me or there is something wrong there.
Stef






Reply | Threaded
Open this post in threaded view
|

Re: Raw pane on byteStrings

Thierry Goubier
Hi Andrei,

Le 24/02/2017 à 17:02, Andrei Chis a écrit :
> Hi Thierry,
>
> Strangely enough I'm getting different times on my machine
> Just to start from the same baseline in a fresh Pharo 60411 image with
> no changes to any of the inspectors over a series of runs I get:
>
> array := (1 to: 1000000) asArray.
> [array inspect] timeToRun.                       "0:00:00:00.145"
> [GTInspector inspect: array] timeToRun. "0:00:00:00.031"

Yes, this is expected. I'm not comparing to the standard EyeInspector,
but with my experiments with a FastTable-derived widget and the
EyeInspector model. GTInspector is a lot faster than the normal
EyeInspector.

As default (the only one displaying 100k elements is the AltInspector)

| array |
array := (1 to: 1000000) asArray.
[AltInspector inspect: array ] timeToRun. "0:00:00:00.096"
[EyeInspector inspect: array] timeToRun.  "0:00:00:00.364"
[GTInspector inspect: array] timeToRun. "0:00:00:00.065"

> If I change indexableDisplayLimit to 50000 and remove the Items view
> then I get:
>
> [GTInspector inspect: array] timeToRun.  "0:00:00:00.124"

[GTInspector inspect: array] timeToRun. "0:00:00:00.256"

So this time is to be compared to the "0:00:00:00.096".

> I'm really interested in seeing what makes GTInspector slower on your
> machine. Did you do other optimizations to SpecInspector?

The optimisations are:
- remove Spec (revert to a pure morphic application),
- use a FastTable-derived widget for a tree view (with a 100k element
limit),
- have the GT views as subitems in the widget.

> I'm using a MacBook Pro Retina - 2.8 GHz Intel Core i7

Back on a Acer Chromebook C720p 1.4GHz Intel Celeron, with vmLatest.

5.0-201702231204  Thu Feb 23 12:36:20 UTC 2017 gcc 4.6.3 [Production
Spur ITHB VM]
CoInterpreter * VMMaker.oscog-EstebanLorenzano.2136 uuid:
40534c32-ca6b-4e97-91ec-31d509e49b0c Feb 23 2017
StackToRegisterMappingCogit * VMMaker.oscog-EstebanLorenzano.2136 uuid:
40534c32-ca6b-4e97-91ec-31d509e49b0c Feb 23 2017
VM: 201702231204 https://github.com/pharo-project/pharo-vm.git $ Date:
Thu Feb 23 13:04:59 2017 +0100 $
Plugins: 201702231204 https://github.com/pharo-project/pharo-vm.git $
Linux testing-docker-b6b0368d-4817-4638-86be-f022b8a58580
4.8.12-040812-generic #201612020431 SMP Fri Dec 2 09:33:31 UTC 2016 i686
i686 i386 GNU/Linux


> Pharo VM version:
> CoInterpreter VMMaker.oscog-HolgerHansPeterFreyther.1880 uuid:
> 16138eb3-2390-40f5-a6c8-15f0494936f8 Oct 10 2016
> StackToRegisterMappingCogit VMMaker.oscog-HolgerHansPeterFreyther.1880
> uuid: 16138eb3-2390-40f5-a6c8-15f0494936f8 Oct 10 2016
> [hidden email]:pharo-project/pharo-vm.git Commit:
> 06744effac0f0aa3b4b32e17636448f9d51d6707 Date: 2016-09-30 08:40:43 +0200
> By: GitHub <[hidden email] <mailto:[hidden email]>>
>
> Cheers,
> Andrei
>
>
> On Fri, Feb 24, 2017 at 3:24 PM, Thierry Goubier
> <[hidden email] <mailto:[hidden email]>> wrote:
>
>
>
>     2017-02-24 14:29 GMT+01:00 Andrei Chis <[hidden email]
>     <mailto:[hidden email]>>:
>
>
>
>         On Fri, Feb 24, 2017 at 12:16 PM, Thierry Goubier
>         <[hidden email] <mailto:[hidden email]>>
>         wrote:
>
>             Hi Andrei,
>
>             2017-02-24 11:31 GMT+01:00 Andrei Chis
>             <[hidden email]
>             <mailto:[hidden email]>>:
>
>                 Hi Thierry,
>
>                 Indeed that's the simplest option now that we are using
>                 fast table.
>
>                 Just now in the case of the Raw view an
>                 OrderedCollection is used to store all displayed elements.
>                 If you display large collections every time you open the
>                 Raw view it will instantiate a collection of size 100k
>                 and instantiate 100k objects of
>                 type GTInspectorIndexedNode. With FastTable we can
>                 lazily load elements so we should be able to remove this
>                 behaviour and the limit. Just we need to make sure it
>                 will play nicely with automatic refresh. There is also
>                 the issue that when expanding an element in the tree if
>                 it's a collection you don't want to expand all elements.
>
>
>             I'm not sure you need to worry too much about that one; in
>             practical experiments, creating that 100k collection for
>             viewing (and the associated nodes instances) isn't too
>             costly (unless creating the GTInspectorIndexedNodes has
>             hidden costs: I've only experimented with the EyeInspector
>             framework).
>
>
>         There should be no hidden costs in GTInspectorIndexedNodes.
>         I made some experiments in the latest Pharo version and opening
>         the Raw view on an array with one million numbers takes around
>         120ms when 100k elements are computed.
>         I'll be curious how much it takes on your machine. To test
>         update indexableDisplayLimit to 50000 in
>         Object>>#gtInspectorVariableNodesIn: and remove the annotation
>         from Collection>>#gtInspectorItemsIn: (so that the Items
>         presentation is not loaded)
>
>         arrayLarge := (1 to: 1000000) asArray.
>         arrayLarge inspect.
>
>
>     This is the values I get on my work laptop (core i3-2350M 2.30 Ghz);
>     both inspectors displays 100k elements.
>
>     (1 to: 1000000) asArray in: [ :s |  [s inspect] timeToRun]
>     0:00:00:00.064
>     (1 to: 1000000) asArray in: [:s | [GTInspector inspect: s]
>     timeToRun] 0:00:00:00.381
>
>     Pharo 6 60411
>
>
>
>
>
>             Opening the tree with all elements works fine in my
>             experiments. Tuning scrolling as done in Bloc is necessary,
>             however.
>
>
>
>                 I'm looking now on a lazy data source for FastTable that
>                 plays nicely with GTInspector. Let's see how it will go.
>                 Help is always welcomed :)
>
>
>             As I said: do not overoptimize that part... just remove that
>             limitation on the raw view and measure.
>
>
>         If I measure the Items view on the previous array it takes
>         around 35ms.
>         What I'd like to have is the same opening time for the Items
>         view on large Sets and Dictionaries.
>
>
>     On my machine, the experiment is that displaying the set is fast,
>     but the system becomes totally unresponsive... which may be an issue
>     with the self refresh of the inspector. Yes, it was the culprit.
>
>     (1 to: 1000000) asSet in: [ :s |  [s inspect] timeToRun] 0:00:00:00.034
>     (1 to: 1000000) asSet in: [:s | [GTInspector inspect: s] timeToRun]
>     0:00:00:00.199
>
>     Regards,
>
>     Thierry
>
>
>
>
>
>
>
>                 I think I used the word paginator in the wrong way. If
>                 you have a very large collection (millions of elements)
>                 I do not want to scroll through elements but most likely
>                 jump to a certain element or view just a subset of all
>                 elements. Not really add a paginator like in web pages.
>
>
>             Ok, millions of elements is a bit out of my scope... I'll
>             look for filtering then.
>
>
>         Yes, definitely filtering is the way to go there :)
>
>         Cheers,
>         Andrei
>
>
>
>             Regards,
>
>             Thierry
>
>
>
>                 Cheers,
>                 Andrei
>
>                 On Fri, Feb 24, 2017 at 10:37 AM, Thierry Goubier
>                 <[hidden email]
>                 <mailto:[hidden email]>> wrote:
>
>                     Hi Andrei,
>
>                     if you're using fasttable for the Raw view, you
>                     should be able to reach one 100k elements without
>                     issues. I did some experiments and it handles the
>                     load very well.
>
>                     Avoid the paginator at any cost. This thing is
>                     really user-unfriendly.
>
>                     Regards,
>
>                     Thierry
>
>                     2017-02-23 20:19 GMT+01:00 Andrei Chis
>                     <[hidden email]
>                     <mailto:[hidden email]>>:
>
>                         Hi Stef,
>
>                         Currently that's the default behaviour of the
>                         Raw view: it displays for collections only the
>                         first and the last 21 elements. The Items view
>                         however always should display all the elements
>                         of a collection.
>
>                         The main problem with the Raw view in Pharo 5 is
>                         the speed. In Pharo 6 now the speed of the Raw
>                         view is greately improved so we could increase
>                         those limits. Still for now there should still
>                         be some limit for the Raw view. Ideally we
>                         should add a small widget, something like a
>                         paginator, for navigating through large and very
>                         large collections.
>
>                         Cheers,
>                         Andrei
>
>                         On Feb 23, 2017 19:35, "stepharong"
>                         <[hidden email] <mailto:[hidden email]>>
>                         wrote:
>
>                             Hi
>
>                             I'm trying to debug citezen generation and I
>                             have to compare strings.
>                             Now I think that the raw views (in Pharo 50)
>                             is not good because we cannot see all the
>                             items in raw format.
>                             See the attachements. It jumps from 21 to
>                             174 ...
>                             and what I want to see is of course in the
>                             middle.
>
>                             Is it me or there is something wrong there.
>                             Stef
>
>
>
>
>
>
>


Reply | Threaded
Open this post in threaded view
|

Re: Raw pane on byteStrings

Andrei Chis
Hi Thierry,

Thanks for the measurements. Things are more clear now :)

On Fri, Feb 24, 2017 at 11:52 PM, Thierry Goubier <[hidden email]> wrote:
Hi Andrei,

Le 24/02/2017 à 17:02, Andrei Chis a écrit :
Hi Thierry,

Strangely enough I'm getting different times on my machine
Just to start from the same baseline in a fresh Pharo 60411 image with
no changes to any of the inspectors over a series of runs I get:

array := (1 to: 1000000) asArray.
[array inspect] timeToRun.                       "0:00:00:00.145"
[GTInspector inspect: array] timeToRun. "0:00:00:00.031"

Yes, this is expected. I'm not comparing to the standard EyeInspector, but with my experiments with a FastTable-derived widget and the EyeInspector model. GTInspector is a lot faster than the normal EyeInspector.

As default (the only one displaying 100k elements is the AltInspector)

| array |
array := (1 to: 1000000) asArray.
[AltInspector inspect: array ] timeToRun. "0:00:00:00.096"
[EyeInspector inspect: array] timeToRun.  "0:00:00:00.364"
[GTInspector inspect: array] timeToRun. "0:00:00:00.065"

Indeed this is what I expected.
 


If I change indexableDisplayLimit to 50000 and remove the Items view
then I get:

[GTInspector inspect: array] timeToRun.  "0:00:00:00.124"

[GTInspector inspect: array] timeToRun. "0:00:00:00.256"

So this time is to be compared to the "0:00:00:00.096".

Can you just also try to remove the Items view and set indexableDisplayLimit to 50.
I think there is some small performance degradation in the current tree data source used by Glamour but I'm not 100% sure.



I'm really interested in seeing what makes GTInspector slower on your
machine. Did you do other optimizations to SpecInspector?

The optimisations are:
- remove Spec (revert to a pure morphic application),
- use a FastTable-derived widget for a tree view (with a 100k element limit),
- have the GT views as subitems in the widget.

Nice. Would be also interesting to see if in the future we can merge more the models used by GTInspector and EyeInspector.

For now, for the Raw view I set the limit to 5000 elements. This seems fast/large enough and it allows to add the automatic refresh back to the Raw view. 

Cheers,
Andrei
 


I'm using a MacBook Pro Retina - 2.8 GHz Intel Core i7

Back on a Acer Chromebook C720p 1.4GHz Intel Celeron, with vmLatest.

5.0-201702231204  Thu Feb 23 12:36:20 UTC 2017 gcc 4.6.3 [Production Spur ITHB VM]
CoInterpreter * VMMaker.oscog-EstebanLorenzano.2136 uuid: 40534c32-ca6b-4e97-91ec-31d509e49b0c Feb 23 2017
StackToRegisterMappingCogit * VMMaker.oscog-EstebanLorenzano.2136 uuid: 40534c32-ca6b-4e97-91ec-31d509e49b0c Feb 23 2017
VM: 201702231204 https://github.com/pharo-project/pharo-vm.git $ Date: Thu Feb 23 13:04:59 2017 +0100 $
Plugins: 201702231204 https://github.com/pharo-project/pharo-vm.git $
Linux testing-docker-b6b0368d-4817-4638-86be-f022b8a58580 4.8.12-040812-generic #201612020431 SMP Fri Dec 2 09:33:31 UTC 2016 i686 i686 i386 GNU/Linux


Pharo VM version:
CoInterpreter VMMaker.oscog-HolgerHansPeterFreyther.1880 uuid:
16138eb3-2390-40f5-a6c8-15f0494936f8 Oct 10 2016
StackToRegisterMappingCogit VMMaker.oscog-HolgerHansPeterFreyther.1880
uuid: 16138eb3-2390-40f5-a6c8-15f0494936f8 Oct 10 2016
[hidden email]:pharo-project/pharo-vm.git Commit:
06744effac0f0aa3b4b32e17636448f9d51d6707 Date: 2016-09-30 08:40:43 +0200
By: GitHub <[hidden email] <mailto:[hidden email]>>

Cheers,
Andrei


On Fri, Feb 24, 2017 at 3:24 PM, Thierry Goubier
<[hidden email] <mailto:[hidden email]>> wrote:



    2017-02-24 14:29 GMT+01:00 Andrei Chis <[hidden email]
    <mailto:[hidden email]>>:



        On Fri, Feb 24, 2017 at 12:16 PM, Thierry Goubier
        <[hidden email] <mailto:[hidden email]>>
        wrote:

            Hi Andrei,

            2017-02-24 11:31 GMT+01:00 Andrei Chis
            <[hidden email]
            <mailto:[hidden email]>>:


                Hi Thierry,

                Indeed that's the simplest option now that we are using
                fast table.

                Just now in the case of the Raw view an
                OrderedCollection is used to store all displayed elements.
                If you display large collections every time you open the
                Raw view it will instantiate a collection of size 100k
                and instantiate 100k objects of
                type GTInspectorIndexedNode. With FastTable we can
                lazily load elements so we should be able to remove this
                behaviour and the limit. Just we need to make sure it
                will play nicely with automatic refresh. There is also
                the issue that when expanding an element in the tree if
                it's a collection you don't want to expand all elements.


            I'm not sure you need to worry too much about that one; in
            practical experiments, creating that 100k collection for
            viewing (and the associated nodes instances) isn't too
            costly (unless creating the GTInspectorIndexedNodes has
            hidden costs: I've only experimented with the EyeInspector
            framework).


        There should be no hidden costs in GTInspectorIndexedNodes.
        I made some experiments in the latest Pharo version and opening
        the Raw view on an array with one million numbers takes around
        120ms when 100k elements are computed.
        I'll be curious how much it takes on your machine. To test
        update indexableDisplayLimit to 50000 in
        Object>>#gtInspectorVariableNodesIn: and remove the annotation
        from Collection>>#gtInspectorItemsIn: (so that the Items
        presentation is not loaded)

        arrayLarge := (1 to: 1000000) asArray.
        arrayLarge inspect.


    This is the values I get on my work laptop (core i3-2350M 2.30 Ghz);
    both inspectors displays 100k elements.

    (1 to: 1000000) asArray in: [ :s |  [s inspect] timeToRun]
    0:00:00:00.064
    (1 to: 1000000) asArray in: [:s | [GTInspector inspect: s]
    timeToRun] 0:00:00:00.381

    Pharo 6 60411





            Opening the tree with all elements works fine in my
            experiments. Tuning scrolling as done in Bloc is necessary,
            however.



                I'm looking now on a lazy data source for FastTable that
                plays nicely with GTInspector. Let's see how it will go.
                Help is always welcomed :)


            As I said: do not overoptimize that part... just remove that
            limitation on the raw view and measure.


        If I measure the Items view on the previous array it takes
        around 35ms.
        What I'd like to have is the same opening time for the Items
        view on large Sets and Dictionaries.


    On my machine, the experiment is that displaying the set is fast,
    but the system becomes totally unresponsive... which may be an issue
    with the self refresh of the inspector. Yes, it was the culprit.

    (1 to: 1000000) asSet in: [ :s |  [s inspect] timeToRun] 0:00:00:00.034
    (1 to: 1000000) asSet in: [:s | [GTInspector inspect: s] timeToRun]
    0:00:00:00.199

    Regards,

    Thierry







                I think I used the word paginator in the wrong way. If
                you have a very large collection (millions of elements)
                I do not want to scroll through elements but most likely
                jump to a certain element or view just a subset of all
                elements. Not really add a paginator like in web pages.


            Ok, millions of elements is a bit out of my scope... I'll
            look for filtering then.


        Yes, definitely filtering is the way to go there :)

        Cheers,
        Andrei



            Regards,

            Thierry



                Cheers,
                Andrei

                On Fri, Feb 24, 2017 at 10:37 AM, Thierry Goubier
                <[hidden email]
                <mailto:[hidden email]>> wrote:

                    Hi Andrei,

                    if you're using fasttable for the Raw view, you
                    should be able to reach one 100k elements without
                    issues. I did some experiments and it handles the
                    load very well.

                    Avoid the paginator at any cost. This thing is
                    really user-unfriendly.

                    Regards,

                    Thierry

                    2017-02-23 20:19 GMT+01:00 Andrei Chis
                    <[hidden email]
                    <mailto:[hidden email]>>:

                        Hi Stef,

                        Currently that's the default behaviour of the
                        Raw view: it displays for collections only the
                        first and the last 21 elements. The Items view
                        however always should display all the elements
                        of a collection.

                        The main problem with the Raw view in Pharo 5 is
                        the speed. In Pharo 6 now the speed of the Raw
                        view is greately improved so we could increase
                        those limits. Still for now there should still
                        be some limit for the Raw view. Ideally we
                        should add a small widget, something like a
                        paginator, for navigating through large and very
                        large collections.

                        Cheers,
                        Andrei

                        On Feb 23, 2017 19:35, "stepharong"
                        <[hidden email] <mailto:[hidden email]>>
                        wrote:

                            Hi

                            I'm trying to debug citezen generation and I
                            have to compare strings.
                            Now I think that the raw views (in Pharo 50)
                            is not good because we cannot see all the
                            items in raw format.
                            See the attachements. It jumps from 21 to
                            174 ...
                            and what I want to see is of course in the
                            middle.

                            Is it me or there is something wrong there.
                            Stef










Reply | Threaded
Open this post in threaded view
|

Re: Raw pane on byteStrings

Andrei Chis
In reply to this post by Thierry Goubier
Hi Thierry,

On Fri, Feb 24, 2017 at 3:49 PM, Thierry Goubier <[hidden email]> wrote:


2017-02-24 15:43 GMT+01:00 Andrei Chis <[hidden email]>:


On Fri, Feb 24, 2017 at 3:28 PM, Denis Kudriashov <[hidden email]> wrote:

2017-02-24 14:29 GMT+01:00 Andrei Chis <[hidden email]>:
There should be no hidden costs in GTInspectorIndexedNodes.
I made some experiments in the latest Pharo version and opening the Raw view on an array with one million numbers takes around 120ms when 100k elements are computed.
I'll be curious how much it takes on your machine. To test update indexableDisplayLimit to 50000 in Object>>#gtInspectorVariableNodesIn: and remove the annotation from Collection>>#gtInspectorItemsIn: (so that the Items presentation is not loaded)

arrayLarge := (1 to: 1000000) asArray.

I really wondering why anybody want instantiate wrapper objects for all array items? Fast table approach is to not do that. Only visible part of items should be recreated

This is how the inspector works for the Items view for Array objects. It also does not use any wrappers and only computes visible part.


As I said, it brings in a lazy data source adapted to arrays in a case where it may well be not needed. My experiments show that there is no need for this optimisation, at least for arrays ~ 100k.

For this particular case there is no special lazy data source. It's just the default table data source from Glamour. In works in this particular case because for a SequenceableCollection we currently have no wrapping of elements, so we can just extract them directly.

Will be worth looking for a lazy data source for a remote inspector but not for Pharo 6.

Cheers,
Andrei
 

Thierry





Reply | Threaded
Open this post in threaded view
|

Re: Raw pane on byteStrings

Thierry Goubier
In reply to this post by Andrei Chis
Hi Andrei,

Le 26/02/2017 à 23:48, Andrei Chis a écrit :

> Hi Thierry,
>
> Thanks for the measurements. Things are more clear now :)
>
> On Fri, Feb 24, 2017 at 11:52 PM, Thierry Goubier
> <[hidden email] <mailto:[hidden email]>> wrote:
>
>     Hi Andrei,
>
>     Le 24/02/2017 à 17:02, Andrei Chis a écrit :
>
>         Hi Thierry,
>
>         Strangely enough I'm getting different times on my machine
>         Just to start from the same baseline in a fresh Pharo 60411
>         image with
>         no changes to any of the inspectors over a series of runs I get:
>
>         array := (1 to: 1000000) asArray.
>         [array inspect] timeToRun.                       "0:00:00:00.145"
>         [GTInspector inspect: array] timeToRun. "0:00:00:00.031"
>
>
>     Yes, this is expected. I'm not comparing to the standard
>     EyeInspector, but with my experiments with a FastTable-derived
>     widget and the EyeInspector model. GTInspector is a lot faster than
>     the normal EyeInspector.
>
>     As default (the only one displaying 100k elements is the AltInspector)
>
>     | array |
>     array := (1 to: 1000000) asArray.
>     [AltInspector inspect: array ] timeToRun. "0:00:00:00.096"
>     [EyeInspector inspect: array] timeToRun.  "0:00:00:00.364"
>     [GTInspector inspect: array] timeToRun. "0:00:00:00.065"
>
>
> Indeed this is what I expected.
>
>
>
>
>         If I change indexableDisplayLimit to 50000 and remove the Items view
>         then I get:
>
>         [GTInspector inspect: array] timeToRun.  "0:00:00:00.124"
>
>
>     [GTInspector inspect: array] timeToRun. "0:00:00:00.256"
>
>     So this time is to be compared to the "0:00:00:00.096".
>
>
> Can you just also try to remove the Items view and set
> indexableDisplayLimit to 50.

In the previous times, the Item view was removed. If
indexableDisplayLimit is set at 50, then the time become:

[GTInspector inspect: array] timeToRun. "0:00:00:00.156"

Which is interesting in itself: multiplying the limit by 1000 (from 50
to 50k) increase the time by a factor of ~1.6.

> I think there is some small performance degradation in the current tree
> data source used by Glamour but I'm not 100% sure.

It looks pretty efficient to me. There isn't much increase in times.

I looked at the scaling in my experiment: the lower bound in opening the
inspector is reached for a 100k elements array. Anything smaller
has the same time (~80ms). The 1000k array increase that by ~15/20ms.

>         I'm really interested in seeing what makes GTInspector slower on
>         your
>         machine. Did you do other optimizations to SpecInspector?
>
>
>     The optimisations are:
>     - remove Spec (revert to a pure morphic application),
>     - use a FastTable-derived widget for a tree view (with a 100k
>     element limit),
>     - have the GT views as subitems in the widget.
>
>
> Nice. Would be also interesting to see if in the future we can merge
> more the models used by GTInspector and EyeInspector.

Could be. I just used them out of convenience, and I needed an inspector
with the ability to view large trees.

> For now, for the Raw view I set the limit to 5000 elements. This seems
> fast/large enough and it allows to add the automatic refresh back to the
> Raw view.

I wonder if it wouldn't be possible to tune the automatic refresh.
Increase the time interval if it takes to long (for example when
refreshing a set with 1M elements). But 5000 elements is already a good
limit.

Regards,

Thierry

> Cheers,
> Andrei
>
>
>
>
>         I'm using a MacBook Pro Retina - 2.8 GHz Intel Core i7
>
>
>     Back on a Acer Chromebook C720p 1.4GHz Intel Celeron, with vmLatest.
>
>     5.0-201702231204  Thu Feb 23 12:36:20 UTC 2017 gcc 4.6.3 [Production
>     Spur ITHB VM]
>     CoInterpreter * VMMaker.oscog-EstebanLorenzano.2136 uuid:
>     40534c32-ca6b-4e97-91ec-31d509e49b0c Feb 23 2017
>     StackToRegisterMappingCogit * VMMaker.oscog-EstebanLorenzano.2136
>     uuid: 40534c32-ca6b-4e97-91ec-31d509e49b0c Feb 23 2017
>     VM: 201702231204 https://github.com/pharo-project/pharo-vm.git
>     <https://github.com/pharo-project/pharo-vm.git> $ Date: Thu Feb 23
>     13:04:59 2017 +0100 $
>     Plugins: 201702231204 https://github.com/pharo-project/pharo-vm.git
>     <https://github.com/pharo-project/pharo-vm.git> $
>     Linux testing-docker-b6b0368d-4817-4638-86be-f022b8a58580
>     4.8.12-040812-generic #201612020431 SMP Fri Dec 2 09:33:31 UTC 2016
>     i686 i686 i386 GNU/Linux
>
>
>         Pharo VM version:
>         CoInterpreter VMMaker.oscog-HolgerHansPeterFreyther.1880 uuid:
>         16138eb3-2390-40f5-a6c8-15f0494936f8 Oct 10 2016
>         StackToRegisterMappingCogit
>         VMMaker.oscog-HolgerHansPeterFreyther.1880
>         uuid: 16138eb3-2390-40f5-a6c8-15f0494936f8 Oct 10 2016
>         [hidden email]:pharo-project/pharo-vm.git Commit:
>         06744effac0f0aa3b4b32e17636448f9d51d6707 Date: 2016-09-30
>         08:40:43 +0200
>         By: GitHub <[hidden email] <mailto:[hidden email]>
>         <mailto:[hidden email] <mailto:[hidden email]>>>
>
>         Cheers,
>         Andrei
>
>
>         On Fri, Feb 24, 2017 at 3:24 PM, Thierry Goubier
>         <[hidden email] <mailto:[hidden email]>
>         <mailto:[hidden email]
>         <mailto:[hidden email]>>> wrote:
>
>
>
>             2017-02-24 14:29 GMT+01:00 Andrei Chis
>         <[hidden email] <mailto:[hidden email]>
>             <mailto:[hidden email]
>         <mailto:[hidden email]>>>:
>
>
>
>                 On Fri, Feb 24, 2017 at 12:16 PM, Thierry Goubier
>                 <[hidden email]
>         <mailto:[hidden email]>
>         <mailto:[hidden email]
>         <mailto:[hidden email]>>>
>                 wrote:
>
>                     Hi Andrei,
>
>                     2017-02-24 11:31 GMT+01:00 Andrei Chis
>                     <[hidden email]
>         <mailto:[hidden email]>
>                     <mailto:[hidden email]
>         <mailto:[hidden email]>>>:
>
>
>                         Hi Thierry,
>
>                         Indeed that's the simplest option now that we
>         are using
>                         fast table.
>
>                         Just now in the case of the Raw view an
>                         OrderedCollection is used to store all displayed
>         elements.
>                         If you display large collections every time you
>         open the
>                         Raw view it will instantiate a collection of
>         size 100k
>                         and instantiate 100k objects of
>                         type GTInspectorIndexedNode. With FastTable we can
>                         lazily load elements so we should be able to
>         remove this
>                         behaviour and the limit. Just we need to make
>         sure it
>                         will play nicely with automatic refresh. There
>         is also
>                         the issue that when expanding an element in the
>         tree if
>                         it's a collection you don't want to expand all
>         elements.
>
>
>                     I'm not sure you need to worry too much about that
>         one; in
>                     practical experiments, creating that 100k collection for
>                     viewing (and the associated nodes instances) isn't too
>                     costly (unless creating the GTInspectorIndexedNodes has
>                     hidden costs: I've only experimented with the
>         EyeInspector
>                     framework).
>
>
>                 There should be no hidden costs in GTInspectorIndexedNodes.
>                 I made some experiments in the latest Pharo version and
>         opening
>                 the Raw view on an array with one million numbers takes
>         around
>                 120ms when 100k elements are computed.
>                 I'll be curious how much it takes on your machine. To test
>                 update indexableDisplayLimit to 50000 in
>                 Object>>#gtInspectorVariableNodesIn: and remove the
>         annotation
>                 from Collection>>#gtInspectorItemsIn: (so that the Items
>                 presentation is not loaded)
>
>                 arrayLarge := (1 to: 1000000) asArray.
>                 arrayLarge inspect.
>
>
>             This is the values I get on my work laptop (core i3-2350M
>         2.30 Ghz);
>             both inspectors displays 100k elements.
>
>             (1 to: 1000000) asArray in: [ :s |  [s inspect] timeToRun]
>             0:00:00:00.064
>             (1 to: 1000000) asArray in: [:s | [GTInspector inspect: s]
>             timeToRun] 0:00:00:00.381
>
>             Pharo 6 60411
>
>
>
>
>
>                     Opening the tree with all elements works fine in my
>                     experiments. Tuning scrolling as done in Bloc is
>         necessary,
>                     however.
>
>
>
>                         I'm looking now on a lazy data source for
>         FastTable that
>                         plays nicely with GTInspector. Let's see how it
>         will go.
>                         Help is always welcomed :)
>
>
>                     As I said: do not overoptimize that part... just
>         remove that
>                     limitation on the raw view and measure.
>
>
>                 If I measure the Items view on the previous array it takes
>                 around 35ms.
>                 What I'd like to have is the same opening time for the Items
>                 view on large Sets and Dictionaries.
>
>
>             On my machine, the experiment is that displaying the set is
>         fast,
>             but the system becomes totally unresponsive... which may be
>         an issue
>             with the self refresh of the inspector. Yes, it was the culprit.
>
>             (1 to: 1000000) asSet in: [ :s |  [s inspect] timeToRun]
>         0:00:00:00.034
>             (1 to: 1000000) asSet in: [:s | [GTInspector inspect: s]
>         timeToRun]
>             0:00:00:00.199
>
>             Regards,
>
>             Thierry
>
>
>
>
>
>
>
>                         I think I used the word paginator in the wrong
>         way. If
>                         you have a very large collection (millions of
>         elements)
>                         I do not want to scroll through elements but
>         most likely
>                         jump to a certain element or view just a subset
>         of all
>                         elements. Not really add a paginator like in web
>         pages.
>
>
>                     Ok, millions of elements is a bit out of my scope...
>         I'll
>                     look for filtering then.
>
>
>                 Yes, definitely filtering is the way to go there :)
>
>                 Cheers,
>                 Andrei
>
>
>
>                     Regards,
>
>                     Thierry
>
>
>
>                         Cheers,
>                         Andrei
>
>                         On Fri, Feb 24, 2017 at 10:37 AM, Thierry Goubier
>                         <[hidden email]
>         <mailto:[hidden email]>
>                         <mailto:[hidden email]
>         <mailto:[hidden email]>>> wrote:
>
>                             Hi Andrei,
>
>                             if you're using fasttable for the Raw view, you
>                             should be able to reach one 100k elements
>         without
>                             issues. I did some experiments and it
>         handles the
>                             load very well.
>
>                             Avoid the paginator at any cost. This thing is
>                             really user-unfriendly.
>
>                             Regards,
>
>                             Thierry
>
>                             2017-02-23 20:19 GMT+01:00 Andrei Chis
>                             <[hidden email]
>         <mailto:[hidden email]>
>                             <mailto:[hidden email]
>         <mailto:[hidden email]>>>:
>
>                                 Hi Stef,
>
>                                 Currently that's the default behaviour
>         of the
>                                 Raw view: it displays for collections
>         only the
>                                 first and the last 21 elements. The
>         Items view
>                                 however always should display all the
>         elements
>                                 of a collection.
>
>                                 The main problem with the Raw view in
>         Pharo 5 is
>                                 the speed. In Pharo 6 now the speed of
>         the Raw
>                                 view is greately improved so we could
>         increase
>                                 those limits. Still for now there should
>         still
>                                 be some limit for the Raw view. Ideally we
>                                 should add a small widget, something like a
>                                 paginator, for navigating through large
>         and very
>                                 large collections.
>
>                                 Cheers,
>                                 Andrei
>
>                                 On Feb 23, 2017 19:35, "stepharong"
>                                 <[hidden email]
>         <mailto:[hidden email]> <mailto:[hidden email]
>         <mailto:[hidden email]>>>
>                                 wrote:
>
>                                     Hi
>
>                                     I'm trying to debug citezen
>         generation and I
>                                     have to compare strings.
>                                     Now I think that the raw views (in
>         Pharo 50)
>                                     is not good because we cannot see
>         all the
>                                     items in raw format.
>                                     See the attachements. It jumps from
>         21 to
>                                     174 ...
>                                     and what I want to see is of course
>         in the
>                                     middle.
>
>                                     Is it me or there is something wrong
>         there.
>                                     Stef
>
>
>
>
>
>
>
>
>
>


12