inspector performance improvement

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

inspector performance improvement

Tudor Girba-2
Hi,

Andrei and Alex did a great job at improving the performance and scalability of the inspector.

There are a couple of main improvements:
- Added FastTable support for the Raw presentation for all objects. This implied completing the support for the tree presentation binding to fast table.
- Moved the Items presentation for collections to FastTable.
- Improved the rendering of Glamour to no longer rely on the default Morph>>#adoptPaneColor. It turns out that using adoptPaneColor triggers a relayout of the morph, even if it is not visible. We extended PanelMorph in the context of Glamour with a less needy logic.

I will not tell you how fast it is. I would rather want you to play with it :).

The change is already in the Moose image. It is not yet in Pharo, but it will be soon. In the meantime you can be load it in Pharo 6 like this:

        Gofer new
                smalltalkhubUser: 'Moose' project: 'GToolkit';
                package: 'ConfigurationOfGToolkitCore';
                load.
        (#ConfigurationOfGToolkitCore asClass project version: #stable) load

To play with it, try this with both the current version and the new one and the Spec Inspector if you want (just make sure you save the image beforehand):

        collection := (1 to: 100000000) asArray.
        [collection inspect] timeToRun.

        collection := (1 to: 100000000).
        [collection inspect] timeToRun.

(for these two ones notice that Items do not appear at all in the old implementation)

        [World inspect] timeToRun.


There are still a couple of issues open, such as the sorting of the columns. We would need your help with testing this, and report possible missing issues.

Cheers,
Doru


--
www.tudorgirba.com
www.feenk.com

"Obvious things are difficult to teach."





Reply | Threaded
Open this post in threaded view
|

Re: inspector performance improvement

stepharong
Great job!!!!
It feels lot snappier :)
Now it is a real pleasure to navigate objects.
This is nice to see that you are paying attention because the engine is  
getting faster and this is important
that the tools do not eat all the power by accident.
Tx a lot


On Sat, 11 Feb 2017 21:31:56 +0100, Tudor Girba <[hidden email]>  
wrote:

> Hi,
>
> Andrei and Alex did a great job at improving the performance and  
> scalability of the inspector.
>
> There are a couple of main improvements:
> - Added FastTable support for the Raw presentation for all objects. This  
> implied completing the support for the tree presentation binding to fast  
> table.
> - Moved the Items presentation for collections to FastTable.
> - Improved the rendering of Glamour to no longer rely on the default  
> Morph>>#adoptPaneColor. It turns out that using adoptPaneColor triggers  
> a relayout of the morph, even if it is not visible. We extended  
> PanelMorph in the context of Glamour with a less needy logic.
>
> I will not tell you how fast it is. I would rather want you to play with  
> it :).
>
> The change is already in the Moose image. It is not yet in Pharo, but it  
> will be soon. In the meantime you can be load it in Pharo 6 like this:
>
> Gofer new
> smalltalkhubUser: 'Moose' project: 'GToolkit';
> package: 'ConfigurationOfGToolkitCore';
> load.
> (#ConfigurationOfGToolkitCore asClass project version: #stable) load
>
> To play with it, try this with both the current version and the new one  
> and the Spec Inspector if you want (just make sure you save the image  
> beforehand):
>
> collection := (1 to: 100000000) asArray.
> [collection inspect] timeToRun.
>
> collection := (1 to: 100000000).
> [collection inspect] timeToRun.
>
> (for these two ones notice that Items do not appear at all in the old  
> implementation)
>
> [World inspect] timeToRun.
>
>
> There are still a couple of issues open, such as the sorting of the  
> columns. We would need your help with testing this, and report possible  
> missing issues.
>
> Cheers,
> Doru
>
>
> --
> www.tudorgirba.com
> www.feenk.com
>
> "Obvious things are difficult to teach."
>
>
>
>
>


--
Using Opera's mail client: http://www.opera.com/mail/

Reply | Threaded
Open this post in threaded view
|

Re: inspector performance improvement

philippeback
Any chance for this to run on a 5.0?

Phil

On Sun, Feb 12, 2017 at 3:13 PM, stepharong <[hidden email]> wrote:
Great job!!!!
It feels lot snappier :)
Now it is a real pleasure to navigate objects.
This is nice to see that you are paying attention because the engine is getting faster and this is important
that the tools do not eat all the power by accident.
Tx a lot


On Sat, 11 Feb 2017 21:31:56 +0100, Tudor Girba <[hidden email]> wrote:

Hi,

Andrei and Alex did a great job at improving the performance and scalability of the inspector.

There are a couple of main improvements:
- Added FastTable support for the Raw presentation for all objects. This implied completing the support for the tree presentation binding to fast table.
- Moved the Items presentation for collections to FastTable.
- Improved the rendering of Glamour to no longer rely on the default Morph>>#adoptPaneColor. It turns out that using adoptPaneColor triggers a relayout of the morph, even if it is not visible. We extended PanelMorph in the context of Glamour with a less needy logic.

I will not tell you how fast it is. I would rather want you to play with it :).

The change is already in the Moose image. It is not yet in Pharo, but it will be soon. In the meantime you can be load it in Pharo 6 like this:

        Gofer new
                smalltalkhubUser: 'Moose' project: 'GToolkit';
                package: 'ConfigurationOfGToolkitCore';
                load.
        (#ConfigurationOfGToolkitCore asClass project version: #stable) load

To play with it, try this with both the current version and the new one and the Spec Inspector if you want (just make sure you save the image beforehand):

        collection := (1 to: 100000000) asArray.
        [collection inspect] timeToRun.

        collection := (1 to: 100000000).
        [collection inspect] timeToRun.

(for these two ones notice that Items do not appear at all in the old implementation)

        [World inspect] timeToRun.


There are still a couple of issues open, such as the sorting of the columns. We would need your help with testing this, and report possible missing issues.

Cheers,
Doru


--
www.tudorgirba.com
www.feenk.com

"Obvious things are difficult to teach."







--
Using Opera's mail client: http://www.opera.com/mail/


Reply | Threaded
Open this post in threaded view
|

Re: inspector performance improvement

Andrei Chis
On Sun, Feb 12, 2017 at 5:07 PM, [hidden email] <[hidden email]> wrote:
> Any chance for this to run on a 5.0?

There are two main changes: moved inspector to fast table and some
fixes in the glamour renderer.
Moving the changes for the glamour renderer should be straightforward.
I can point you to the list of relevant commits if you want to make a
slice (We'll just need afterwards to also update
ConfigurationOfGlamourCore to still work for Pharo 5)

The FastTable changes could require more work but that should also be
doable. However, now we still have some issues to fix there so that's
not done for the moment.


Cheers,
Andrei


>
> Phil
>
> On Sun, Feb 12, 2017 at 3:13 PM, stepharong <[hidden email]> wrote:
>>
>> Great job!!!!
>> It feels lot snappier :)
>> Now it is a real pleasure to navigate objects.
>> This is nice to see that you are paying attention because the engine is
>> getting faster and this is important
>> that the tools do not eat all the power by accident.
>> Tx a lot
>>
>>
>> On Sat, 11 Feb 2017 21:31:56 +0100, Tudor Girba <[hidden email]>
>> wrote:
>>
>>> Hi,
>>>
>>> Andrei and Alex did a great job at improving the performance and
>>> scalability of the inspector.
>>>
>>> There are a couple of main improvements:
>>> - Added FastTable support for the Raw presentation for all objects. This
>>> implied completing the support for the tree presentation binding to fast
>>> table.
>>> - Moved the Items presentation for collections to FastTable.
>>> - Improved the rendering of Glamour to no longer rely on the default
>>> Morph>>#adoptPaneColor. It turns out that using adoptPaneColor triggers a
>>> relayout of the morph, even if it is not visible. We extended PanelMorph in
>>> the context of Glamour with a less needy logic.
>>>
>>> I will not tell you how fast it is. I would rather want you to play with
>>> it :).
>>>
>>> The change is already in the Moose image. It is not yet in Pharo, but it
>>> will be soon. In the meantime you can be load it in Pharo 6 like this:
>>>
>>>         Gofer new
>>>                 smalltalkhubUser: 'Moose' project: 'GToolkit';
>>>                 package: 'ConfigurationOfGToolkitCore';
>>>                 load.
>>>         (#ConfigurationOfGToolkitCore asClass project version: #stable)
>>> load
>>>
>>> To play with it, try this with both the current version and the new one
>>> and the Spec Inspector if you want (just make sure you save the image
>>> beforehand):
>>>
>>>         collection := (1 to: 100000000) asArray.
>>>         [collection inspect] timeToRun.
>>>
>>>         collection := (1 to: 100000000).
>>>         [collection inspect] timeToRun.
>>>
>>> (for these two ones notice that Items do not appear at all in the old
>>> implementation)
>>>
>>>         [World inspect] timeToRun.
>>>
>>>
>>> There are still a couple of issues open, such as the sorting of the
>>> columns. We would need your help with testing this, and report possible
>>> missing issues.
>>>
>>> Cheers,
>>> Doru
>>>
>>>
>>> --
>>> www.tudorgirba.com
>>> www.feenk.com
>>>
>>> "Obvious things are difficult to teach."
>>>
>>>
>>>
>>>
>>>
>>
>>
>> --
>> Using Opera's mail client: http://www.opera.com/mail/
>>
>

Reply | Threaded
Open this post in threaded view
|

Re: inspector performance improvement

Henrik Sperre Johansen
I guess I'm late to the party, downloaded the latest Moose image today, compared to Pharo5/6, it's night and day!
Browsing large collections is quicker than I can ever remember, if emails had emoticons, there'd be a :partyhat: right here, great job!

You're probably aware already, but there's an issue with the indexes for the items tab when collections contain the same object multiple times, say

o := Object new.
oc := OrderedCollection new: 2.
1 to: 2 do: [:i | oc add: o].
oc inspect

or

ba := (ByteArray new: 20000) inspect.
0 to: 19999 do: [:i | ba at: i + 1 put: i \\ 255 + 1].
ba inspect.

Finding a solution that avoids doing indexOf: lookups, but pass along the index instead when needed would be nice ;)

The second example also highlight that there seems to be a (legacy?) restriction of 10k entries in the raw view;  the list jumps from index 5000 -> 15001, this shouldn't be necessary anymore, right?.

*EDIT* After some digging, and testing with a large collection where the limit is raised, I can see why it's there...
Find attached some improvements to FT that should (at least act as a PoC that it's possible to) improve performance enough with large trees to possibly remove indexableDisplayLimit entirely.
Tested by inspecting a 2M ByteArray, while opening the tab took ~5 secs, navigation/scrolling afterwards was smooth sailing.  

Cheers,
Henry


> On 13 Feb 2017, at 11:56 , Andrei Chis <[hidden email]> wrote:
>
> On Sun, Feb 12, 2017 at 5:07 PM, [hidden email] <[hidden email]> wrote:
>> Any chance for this to run on a 5.0?
>
> There are two main changes: moved inspector to fast table and some
> fixes in the glamour renderer.
> Moving the changes for the glamour renderer should be straightforward.
> I can point you to the list of relevant commits if you want to make a
> slice (We'll just need afterwards to also update
> ConfigurationOfGlamourCore to still work for Pharo 5)
>
> The FastTable changes could require more work but that should also be
> doable. However, now we still have some issues to fix there so that's
> not done for the moment.
>
>
> Cheers,
> Andrei
>
>
>>
>> Phil
>>
>> On Sun, Feb 12, 2017 at 3:13 PM, stepharong <[hidden email]> wrote:
>>>
>>> Great job!!!!
>>> It feels lot snappier :)
>>> Now it is a real pleasure to navigate objects.
>>> This is nice to see that you are paying attention because the engine is
>>> getting faster and this is important
>>> that the tools do not eat all the power by accident.
>>> Tx a lot
>>>
>>>
>>> On Sat, 11 Feb 2017 21:31:56 +0100, Tudor Girba <[hidden email]>
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> Andrei and Alex did a great job at improving the performance and
>>>> scalability of the inspector.
>>>>
>>>> There are a couple of main improvements:
>>>> - Added FastTable support for the Raw presentation for all objects. This
>>>> implied completing the support for the tree presentation binding to fast
>>>> table.
>>>> - Moved the Items presentation for collections to FastTable.
>>>> - Improved the rendering of Glamour to no longer rely on the default
>>>> Morph>>#adoptPaneColor. It turns out that using adoptPaneColor triggers a
>>>> relayout of the morph, even if it is not visible. We extended PanelMorph in
>>>> the context of Glamour with a less needy logic.
>>>>
>>>> I will not tell you how fast it is. I would rather want you to play with
>>>> it :).
>>>>
>>>> The change is already in the Moose image. It is not yet in Pharo, but it
>>>> will be soon. In the meantime you can be load it in Pharo 6 like this:
>>>>
>>>>        Gofer new
>>>>                smalltalkhubUser: 'Moose' project: 'GToolkit';
>>>>                package: 'ConfigurationOfGToolkitCore';
>>>>                load.
>>>>        (#ConfigurationOfGToolkitCore asClass project version: #stable)
>>>> load
>>>>
>>>> To play with it, try this with both the current version and the new one
>>>> and the Spec Inspector if you want (just make sure you save the image
>>>> beforehand):
>>>>
>>>>        collection := (1 to: 100000000) asArray.
>>>>        [collection inspect] timeToRun.
>>>>
>>>>        collection := (1 to: 100000000).
>>>>        [collection inspect] timeToRun.
>>>>
>>>> (for these two ones notice that Items do not appear at all in the old
>>>> implementation)
>>>>
>>>>        [World inspect] timeToRun.
>>>>
>>>>
>>>> There are still a couple of issues open, such as the sorting of the
>>>> columns. We would need your help with testing this, and report possible
>>>> missing issues.
>>>>
>>>> Cheers,
>>>> Doru
>>>>
>>>>
>>>> --
>>>> www.tudorgirba.com
>>>> www.feenk.com
>>>>
>>>> "Obvious things are difficult to teach."
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Using Opera's mail client: http://www.opera.com/mail/
>>>
>>
>


Morphic-Widgets-FastTable-HenrikSperreJohansen.211.mcz (149K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: inspector performance improvement

Andrei Chis
Hi Henry,

On Thu, Mar 23, 2017 at 10:49 PM, Henrik Johansen <[hidden email]> wrote:
I guess I'm late to the party, downloaded the latest Moose image today, compared to Pharo5/6, it's night and day!
Browsing large collections is quicker than I can ever remember, if emails had emoticons, there'd be a :partyhat: right here, great job!

You're probably aware already, but there's an issue with the indexes for the items tab when collections contain the same object multiple times, say

Yes we know this issue. It's actually fixed in Pharo, but it seems that I forgot to port the slice also to the repo of the inspector. Will do that tomorrow.
 

o := Object new.
oc := OrderedCollection new: 2.
1 to: 2 do: [:i | oc add: o].
oc inspect

or

ba := (ByteArray new: 20000) inspect.
0 to: 19999 do: [:i | ba at: i + 1 put: i \\ 255 + 1].
ba inspect.

Finding a solution that avoids doing indexOf: lookups, but pass along the index instead when needed would be nice ;)

That's actually how the view for SequenceableCollection objects works; this problem is related to caching.
 

The second example also highlight that there seems to be a (legacy?) restriction of 10k entries in the raw view;  the list jumps from index 5000 -> 15001, this shouldn't be necessary anymore, right?.

In Pharo 5 the limit was 42 :). We increased it in Pharo 6 to 10k as we thought it is good enough. The 'Items' presentation however always displays the entire number of elements.
 

*EDIT* After some digging, and testing with a large collection where the limit is raised, I can see why it's there...
Find attached some improvements to FT that should (at least act as a PoC that it's possible to) improve performance enough with large trees to possibly remove indexableDisplayLimit entirely.
Tested by inspecting a 2M ByteArray, while opening the tab took ~5 secs, navigation/scrolling afterwards was smooth sailing

It's definitely possible to eliminate the indexableDisplayLimit. One solution is to use a lazy model that only computes visible items for all collection presentations.  I see that your solution plays with #visibleChildren. That's definitely a way to improve the tree widget. We'll look at this more for Pharo 7.

If you made it so far you can also give the automatic refresh of the Raw view a try :) (GTInspector enableStepRefresh).

Cheers,
Andrei
 
Cheers,
Henry


> On 13 Feb 2017, at 11:56 , Andrei Chis <[hidden email]> wrote:
>
> On Sun, Feb 12, 2017 at 5:07 PM, [hidden email] <[hidden email]> wrote:
>> Any chance for this to run on a 5.0?
>
> There are two main changes: moved inspector to fast table and some
> fixes in the glamour renderer.
> Moving the changes for the glamour renderer should be straightforward.
> I can point you to the list of relevant commits if you want to make a
> slice (We'll just need afterwards to also update
> ConfigurationOfGlamourCore to still work for Pharo 5)
>
> The FastTable changes could require more work but that should also be
> doable. However, now we still have some issues to fix there so that's
> not done for the moment.
>
>
> Cheers,
> Andrei
>
>
>>
>> Phil
>>
>> On Sun, Feb 12, 2017 at 3:13 PM, stepharong <[hidden email]> wrote:
>>>
>>> Great job!!!!
>>> It feels lot snappier :)
>>> Now it is a real pleasure to navigate objects.
>>> This is nice to see that you are paying attention because the engine is
>>> getting faster and this is important
>>> that the tools do not eat all the power by accident.
>>> Tx a lot
>>>
>>>
>>> On Sat, 11 Feb 2017 21:31:56 +0100, Tudor Girba <[hidden email]>
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> Andrei and Alex did a great job at improving the performance and
>>>> scalability of the inspector.
>>>>
>>>> There are a couple of main improvements:
>>>> - Added FastTable support for the Raw presentation for all objects. This
>>>> implied completing the support for the tree presentation binding to fast
>>>> table.
>>>> - Moved the Items presentation for collections to FastTable.
>>>> - Improved the rendering of Glamour to no longer rely on the default
>>>> Morph>>#adoptPaneColor. It turns out that using adoptPaneColor triggers a
>>>> relayout of the morph, even if it is not visible. We extended PanelMorph in
>>>> the context of Glamour with a less needy logic.
>>>>
>>>> I will not tell you how fast it is. I would rather want you to play with
>>>> it :).
>>>>
>>>> The change is already in the Moose image. It is not yet in Pharo, but it
>>>> will be soon. In the meantime you can be load it in Pharo 6 like this:
>>>>
>>>>        Gofer new
>>>>                smalltalkhubUser: 'Moose' project: 'GToolkit';
>>>>                package: 'ConfigurationOfGToolkitCore';
>>>>                load.
>>>>        (#ConfigurationOfGToolkitCore asClass project version: #stable)
>>>> load
>>>>
>>>> To play with it, try this with both the current version and the new one
>>>> and the Spec Inspector if you want (just make sure you save the image
>>>> beforehand):
>>>>
>>>>        collection := (1 to: 100000000) asArray.
>>>>        [collection inspect] timeToRun.
>>>>
>>>>        collection := (1 to: 100000000).
>>>>        [collection inspect] timeToRun.
>>>>
>>>> (for these two ones notice that Items do not appear at all in the old
>>>> implementation)
>>>>
>>>>        [World inspect] timeToRun.
>>>>
>>>>
>>>> There are still a couple of issues open, such as the sorting of the
>>>> columns. We would need your help with testing this, and report possible
>>>> missing issues.
>>>>
>>>> Cheers,
>>>> Doru
>>>>
>>>>
>>>> --
>>>> www.tudorgirba.com
>>>> www.feenk.com
>>>>
>>>> "Obvious things are difficult to teach."
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Using Opera's mail client: http://www.opera.com/mail/
>>>
>>
>