[vwnc] LessProgress gives more processes

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

[vwnc] LessProgress gives more processes

Paul Baumann
The LessProgress package in VW 7.5 creates thousands of userBackgroundPriority (P30) processes while loading from StORE.
 
The method Store.StoreProgressView>>clearSoon forks new processes that are intended to get attention after 2.5 seconds. The forked process never gets to the delay because code loading (during #loadPundle) was done at P50 with no suspends. One problem is that more than one process is created. Another problem is that the forked code never has a chance to execute to the delay (to schedule resumption) until after work is done. There are plenty of ways to address this, but I'll leave it to Cincom. This only affects load performance.
 
Sometimes less is more. ;)
 
Paul Baumann 
 
 

This message may contain confidential information and is intended for specific recipients unless explicitly noted otherwise. If you have reason to believe you are not an intended recipient of this message, please delete it and notify the sender. This message may not represent the opinion of IntercontinentalExchange, Inc. (ICE), its subsidiaries or affiliates, and does not constitute a contract or guarantee. Unencrypted electronic mail is not secure and the recipient of this message is expected to provide safeguards from viruses and pursue alternate means of communication where privacy or a binding message is desired.


_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
Reply | Threaded
Open this post in threaded view
|

Re: [vwnc] LessProgress gives more processes

Andres Valloud-6
Paul,
 
AR 52930 subclasses IncrementNotification into DecimatingIncrementNotification, which only honors a request to update the progress dialog once every 10 or so times.  It also refactors all references to IncrementNotification so changing from the normal to the decimating one is easier.  This accomplishes goals similar to that of LessProgress while using the current dialog.
 
Andres.


From: [hidden email] [mailto:[hidden email]] On Behalf Of Paul Baumann
Sent: Wednesday, March 19, 2008 10:33 AM
To: [hidden email]
Subject: [vwnc] LessProgress gives more processes

The LessProgress package in VW 7.5 creates thousands of userBackgroundPriority (P30) processes while loading from StORE.
 
The method Store.StoreProgressView>>clearSoon forks new processes that are intended to get attention after 2.5 seconds. The forked process never gets to the delay because code loading (during #loadPundle) was done at P50 with no suspends. One problem is that more than one process is created. Another problem is that the forked code never has a chance to execute to the delay (to schedule resumption) until after work is done. There are plenty of ways to address this, but I'll leave it to Cincom. This only affects load performance.
 
Sometimes less is more. ;)
 
Paul Baumann 
 
 

This message may contain confidential information and is intended for specific recipients unless explicitly noted otherwise. If you have reason to believe you are not an intended recipient of this message, please delete it and notify the sender. This message may not represent the opinion of IntercontinentalExchange, Inc. (ICE), its subsidiaries or affiliates, and does not constitute a contract or guarantee. Unencrypted electronic mail is not secure and the recipient of this message is expected to provide safeguards from viruses and pursue alternate means of communication where privacy or a binding message is desired.


_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
Reply | Threaded
Open this post in threaded view
|

Re: [vwnc] LessProgress gives more processes

Travis Griggs-3
In reply to this post by Paul Baumann
On Mar 19, 2008, at 10:32 AM, Paul Baumann wrote:

> The LessProgress package in VW 7.5 creates thousands of  
> userBackgroundPriority (P30) processes while loading from StORE.
>
> The method Store.StoreProgressView>>clearSoon forks new processes  
> that are intended to get attention after 2.5 seconds. The forked  
> process never gets to the delay because code loading (during  
> #loadPundle) was done at P50 with no suspends. One problem is that  
> more than one process is created. Another problem is that the forked  
> code never has a chance to execute to the delay (to schedule  
> resumption) until after work is done. There are plenty of ways to  
> address this, but I'll leave it to Cincom. This only affects load  
> performance.

I'm not sure how you got so many extra processes. grep | wc'ing the  
stacks file you sent me shows about 9000 processes. I ran a polling  
loop where I Bag'ed up the result of Process allInstances 10 times a  
second, and only managed to get about 80 extra processes (and who  
knows how many of those just hadn't been GC'ed yet). I did so twice  
while loading all StoreForGlorpVWUI.

Anyway, you are right. What appears to have happened, is that the  
intent of clearSoon was lost in a refactoring. If you look at the  
noteProgress: method (which is no longer used, but not cleaned out  
after the refactoring either), you can see that the clearSoon was only  
to fire when the progress wasDone, not on every update. But the way  
the code was changed, by cutting these up into separate methods, this  
intent was lost and a clearSoon is triggered on every update. Not just  
the finished one.

I've fixed this and published it as 7.6 16. I now see about 10+  
processes through the run. Which I'm willing to assume is just the  
GC's ability to retire them. The same change could be made to the 7.5  
branch. For all I know, the 7.6 branch works on 7.5? Dunno.

>  Sometimes less is more. ;)

Almost always. Even in this case. :)

--
Travis Griggs
Objologist
"I think that we should be men first, and subjects afterward." - Henry  
David Thoreau



_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
Reply | Threaded
Open this post in threaded view
|

Re: [vwnc] LessProgress gives more processes

Paul Baumann
A very large nested bundle of code was being loaded into a clean image.
I'll test if you send me your changes.

Thanks for taking the time to fix this 3rd party addition.

Paul Baumann


-----Original Message-----
From: Travis Griggs [mailto:[hidden email]]
Sent: Wednesday, March 19, 2008 4:45 PM
To: Paul Baumann
Cc: [hidden email]
Subject: Re: [vwnc] LessProgress gives more processes
Importance: High

On Mar 19, 2008, at 10:32 AM, Paul Baumann wrote:
> The LessProgress package in VW 7.5 creates thousands of
> userBackgroundPriority (P30) processes while loading from StORE.
>
> The method Store.StoreProgressView>>clearSoon forks new processes that

> are intended to get attention after 2.5 seconds. The forked process
> never gets to the delay because code loading (during
> #loadPundle) was done at P50 with no suspends. One problem is that
> more than one process is created. Another problem is that the forked
> code never has a chance to execute to the delay (to schedule
> resumption) until after work is done. There are plenty of ways to
> address this, but I'll leave it to Cincom. This only affects load
> performance.

I'm not sure how you got so many extra processes. grep | wc'ing the
stacks file you sent me shows about 9000 processes. I ran a polling loop
where I Bag'ed up the result of Process allInstances 10 times a second,
and only managed to get about 80 extra processes (and who knows how many
of those just hadn't been GC'ed yet). I did so twice while loading all
StoreForGlorpVWUI.

Anyway, you are right. What appears to have happened, is that the intent
of clearSoon was lost in a refactoring. If you look at the
noteProgress: method (which is no longer used, but not cleaned out after
the refactoring either), you can see that the clearSoon was only to fire
when the progress wasDone, not on every update. But the way the code was
changed, by cutting these up into separate methods, this intent was lost
and a clearSoon is triggered on every update. Not just the finished one.

I've fixed this and published it as 7.6 16. I now see about 10+
processes through the run. Which I'm willing to assume is just the GC's
ability to retire them. The same change could be made to the 7.5 branch.
For all I know, the 7.6 branch works on 7.5? Dunno.

>  Sometimes less is more. ;)

Almost always. Even in this case. :)

--
Travis Griggs
Objologist
"I think that we should be men first, and subjects afterward." - Henry
David Thoreau
 
--------------------------------------------------------
This message may contain confidential information and is intended for specific recipients unless explicitly noted otherwise. If you have reason to believe you are not an intended recipient of this message, please delete it and notify the sender. This message may not represent the opinion of IntercontinentalExchange, Inc. (ICE), its subsidiaries or affiliates, and does not constitute a contract or guarantee. Unencrypted electronic mail is not secure and the recipient of this message is expected to provide safeguards from viruses and pursue alternate means of communication where privacy or a binding message is desired.

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc