Parcel loading taking too long
Locked
Parcel loading taking too long
 
|
Is there anything new here now available that I may do to load parcels in
a "runtime" context faster. What I mean is that we load our parcels at runtime onto a generic SRE and now want to reduce the load time. thanks in advance, -- Charles A. Monteiro http://wiki.nycsmalltalk.org http://www.monteirosfusion.com |
Re: Parcel loading taking too long
|
|
The best speedup is to load parcels and then save your Image.
Also it is worth while looking at the Parcel pre/postLoad blocks as well as class initialization code. Loading multi megabyte parcel takes few seconds for me, but some initialization code may take minutes - in my case it loads external dlls and tries to populate a lot of places which could be populated later. However it is not an issue for me, because I save image, strip and compress, and deliver it this way. Hope this helps, --Mark Charles A. Monteiro wrote: > Is there anything new here now available that I may do to load parcels > in a "runtime" context faster. What I mean is that we load our parcels > at runtime onto a generic SRE and now want to reduce the load time. > > thanks in advance, > > |
|
|
That's how I ship BottomFeeder - I have 35 parcels that can
potentially be updated at runtime, but they are all pre-loaded into the deployed image. If I make an update available, they are re-loaded, either on the fly or at startup. At 02:11 PM 3/7/2007, you wrote: >The best speedup is to load parcels and then save your Image. >Also it is worth while looking at the Parcel pre/postLoad blocks as >well as class initialization code. > <Talk Small and Carry a Big Class Library> James Robertson, Product Manager, Cincom Smalltalk http://www.cincomsmalltalk.com/blog/blogView |
Re: Parcel loading taking too long
|
|
In reply to this post by Mark Pirogovsky-3
well, when the best answer is stop what you are doing :)
there are reasons why I want to load parcels dynamically in our case our now 7 mb SRE which now includes COM support, which could be leaned out, loads about 3 mb of compressed parcels in about 15 secs. So the 15 seconds includes: image startup of 7mb SRE decompressing of parcels which has been timed at 1.2 seconds the loading of the parcels an artifical delay of 500 milliseconds this is on a P4 . 2.4 ghz machine not much initialization been done at parcel loads either On Wed, 07 Mar 2007 14:11:26 -0500, Mark Pirogovsky <[hidden email]> wrote: > The best speedup is to load parcels and then save your Image. > Also it is worth while looking at the Parcel pre/postLoad blocks as well > as class initialization code. > > Loading multi megabyte parcel takes few seconds for me, but some > initialization code may take minutes - in my case it loads external dlls > and tries to populate a lot of places which could be populated later. > However it is not an issue for me, because I save image, strip and > compress, and deliver it this way. > > Hope this helps, > > --Mark > > Charles A. Monteiro wrote: > >> Is there anything new here now available that I may do to load parcels >> in a "runtime" context faster. What I mean is that we load our parcels >> at runtime onto a generic SRE and now want to reduce the load time. >> thanks in advance, >> -- Charles A. Monteiro http://wiki.nycsmalltalk.org http://www.monteirosfusion.com http://monteirofusion.blogspot.com |
Re: Parcel loading taking too long
|
|
Charles,
I have spent some time in the past looking to expedite parcel loading. It appears that parcels are trying to do more then just load your binary code into the image(like maintaining changes and overrides. Also of note: the load slows down significantly if any of the parcels are modifying Classes/methods, which are already loaded into the image. I think the fastest you can do is to load your code using BOSS. --Mark Charles A. Monteiro wrote: > well, when the best answer is stop what you are doing :) > there are reasons why I want to load parcels dynamically > > in our case our now 7 mb SRE which now includes COM support, which > could be leaned out, loads about 3 mb of compressed parcels in about 15 > secs. > > So the 15 seconds includes: > > image startup of 7mb SRE > decompressing of parcels which has been timed at 1.2 seconds > the loading of the parcels > an artifical delay of 500 milliseconds > > this is on a P4 . 2.4 ghz machine > > not much initialization been done at parcel loads either > > On Wed, 07 Mar 2007 14:11:26 -0500, Mark Pirogovsky > <[hidden email]> wrote: > >> The best speedup is to load parcels and then save your Image. >> Also it is worth while looking at the Parcel pre/postLoad blocks as >> well as class initialization code. >> >> Loading multi megabyte parcel takes few seconds for me, but some >> initialization code may take minutes - in my case it loads external >> dlls and tries to populate a lot of places which could be populated >> later. However it is not an issue for me, because I save image, strip >> and compress, and deliver it this way. >> >> Hope this helps, >> >> --Mark >> >> Charles A. Monteiro wrote: >> >>> Is there anything new here now available that I may do to load >>> parcels in a "runtime" context faster. What I mean is that we load >>> our parcels at runtime onto a generic SRE and now want to reduce >>> the load time. >>> thanks in advance, >>> > > > |
|
|
I've profiled loading via BOSS and via Parcel; BOSS is slower
At 03:37 PM 3/7/2007, you wrote: >Charles, > >I have spent some time in the past looking to expedite parcel loading. >It appears that parcels are trying to do more then just load your >binary code into the image(like maintaining changes and overrides. >Also of note: the load slows down significantly if any of the >parcels are modifying Classes/methods, which are already loaded >into the image. > >I think the fastest you can do is to load your code using BOSS. > >--Mark > >Charles A. Monteiro wrote: > >>well, when the best answer is stop what you are doing :) >>there are reasons why I want to load parcels dynamically >>in our case our now 7 mb SRE which now includes COM support, which >>could be leaned out, loads about 3 mb of compressed parcels in about 15 secs. >>So the 15 seconds includes: >>image startup of 7mb SRE >>decompressing of parcels which has been timed at 1.2 seconds >>the loading of the parcels >>an artifical delay of 500 milliseconds >>this is on a P4 . 2.4 ghz machine >>not much initialization been done at parcel loads either >>On Wed, 07 Mar 2007 14:11:26 -0500, Mark Pirogovsky >><[hidden email]> wrote: >> >>>The best speedup is to load parcels and then save your Image. >>>Also it is worth while looking at the Parcel pre/postLoad blocks >>>as well as class initialization code. >>> >>>Loading multi megabyte parcel takes few seconds for me, but some >>>initialization code may take minutes - in my case it loads >>>external dlls and tries to populate a lot of places which could >>>be populated later. However it is not an issue for me, because I >>>save image, strip and compress, and deliver it this way. >>> >>>Hope this helps, >>> >>>--Mark >>> >>>Charles A. Monteiro wrote: >>> >>>>Is there anything new here now available that I may do to load >>>>parcels in a "runtime" context faster. What I mean is that we >>>>load our parcels at runtime onto a generic SRE and now want to >>>>reduce the load time. >>>> thanks in advance, >> <Talk Small and Carry a Big Class Library> James Robertson, Product Manager, Cincom Smalltalk http://www.cincomsmalltalk.com/blog/blogView |
Re: Parcel loading taking too long
|
|
James Robertson wrote:
> I've profiled loading via BOSS and via Parcel; BOSS is slower > Ok, that is good to know. But is there anything to speed up parcel loading like saving it with some special parameters ? > At 03:37 PM 3/7/2007, you wrote: > >> Charles, >> >> I have spent some time in the past looking to expedite parcel loading. >> It appears that parcels are trying to do more then just load your >> binary code into the image(like maintaining changes and overrides. >> Also of note: the load slows down significantly if any of the parcels >> are modifying Classes/methods, which are already loaded into the image. >> >> I think the fastest you can do is to load your code using BOSS. >> >> --Mark >> >> Charles A. Monteiro wrote: --Mark |
Re: Parcel loading taking too long
|
|
In reply to this post by Charles A. Monteiro-2
On Mar 7, 2007, at 14:56, Charles A. Monteiro wrote:
-- Travis Griggs Objologist "Is success the potential of what could be, or the reality of what is?" |
Re[2]: Parcel loading taking too long
|
|
In reply to this post by Mark Pirogovsky-3
> But is there anything to speed up parcel loading like saving it with
> some special parameters ? There's still the stuff I wrote about a few years back where you disable the system linking as stuff loads. This speeds things up considerably. Hit the archives searching for parcel loading and you'll see a few threads talking about this issue. Cheers, Michael |
Re: Parcel loading taking too long
|
|
Michael Lucas-Smith wrote:
>> But is there anything to speed up parcel loading like saving it with >> some special parameters ? >> > > There's still the stuff I wrote about a few years back where you > disable the system linking as stuff loads. This speeds things up > considerably. Where considerable means that a normal system spends over 40% of its time in relinking when loading from source. (This was measured well over a year ago on our nightly builds.) This is in a situation where we do source loading - binary loading like parcels do might show a bigger slice of time taken by relinking (since I expect they spend less time parsing ). R - > Hit the archives searching for parcel loading and you'll > see a few threads talking about this issue. > > Cheers, > Michael > > > |
Re: Parcel loading taking too long
|
|
In reply to this post by Mark Pirogovsky-3
the runtime is basically a generic image which does not have any of our
code except for some small bootstrapping stuff, however I will double check, there are some shape changes which should already be in the generic engine. The parcel load does too much , I notice that when I was building my OpentalkMatrix stuff. Loading code via Boss in interesting. I remember doing that for external interfaces way back in the 2.x days and even before. thanks -Charles On Wed, 07 Mar 2007 15:37:34 -0500, Mark Pirogovsky <[hidden email]> wrote: > Charles, > > I have spent some time in the past looking to expedite parcel loading. > It appears that parcels are trying to do more then just load your binary > code into the image(like maintaining changes and overrides. > Also of note: the load slows down significantly if any of the parcels > are modifying Classes/methods, which are already loaded into the image. > > I think the fastest you can do is to load your code using BOSS. > > --Mark > > Charles A. Monteiro wrote: > >> well, when the best answer is stop what you are doing :) >> there are reasons why I want to load parcels dynamically >> in our case our now 7 mb SRE which now includes COM support, which >> could be leaned out, loads about 3 mb of compressed parcels in about >> 15 secs. >> So the 15 seconds includes: >> image startup of 7mb SRE >> decompressing of parcels which has been timed at 1.2 seconds >> the loading of the parcels >> an artifical delay of 500 milliseconds >> this is on a P4 . 2.4 ghz machine >> not much initialization been done at parcel loads either >> On Wed, 07 Mar 2007 14:11:26 -0500, Mark Pirogovsky >> <[hidden email]> wrote: >> >>> The best speedup is to load parcels and then save your Image. >>> Also it is worth while looking at the Parcel pre/postLoad blocks as >>> well as class initialization code. >>> >>> Loading multi megabyte parcel takes few seconds for me, but some >>> initialization code may take minutes - in my case it loads external >>> dlls and tries to populate a lot of places which could be populated >>> later. However it is not an issue for me, because I save image, strip >>> and compress, and deliver it this way. >>> >>> Hope this helps, >>> >>> --Mark >>> >>> Charles A. Monteiro wrote: >>> >>>> Is there anything new here now available that I may do to load >>>> parcels in a "runtime" context faster. What I mean is that we load >>>> our parcels at runtime onto a generic SRE and now want to reduce >>>> the load time. >>>> thanks in advance, >>>> >> -- Charles A. Monteiro http://wiki.nycsmalltalk.org http://www.monteirosfusion.com http://monteirofusion.blogspot.com |
Re: Re[2]: Parcel loading taking too long
|
|
In reply to this post by Michael Lucas-Smith
We also use a minimalistic run time image which only contains our run
time configuration and parcel loading framework and we load everything else from parcels. Our profiling suggests the load time is adversely effected by the number of parcels as opposed to the amount of code being loaded. We also use Michael's "stuff" which does indeed make a considerable difference however I feel the load times are still too slow, to the point that encourages composing larger parcels from smaller modules, like creating parcels for entire bundles rather then parcels for each bundle component. This not only complicates deployment but creates an unnatural tensions between development and run time granularity of components. Pre-built or "just in time image" also does not suit environments where one does not have total control of the customer site. For example many sites use Citrix servers where users do not have rights to store cached images and the like. I feel the parcel load time issue needs to be addressed comprehensively if we as a community are serious about shedding our monolithic image approach, SRE or want ST to be a serious contender in scripting environments. If it takes < 3 sec to load my 30mg development image, surely we can load a few hundred kilobytes of additional code in proportional time. I realise an image does not need to compile, re-link or shape change when being loaded but then is it unreasonable to consider parcels as some composable image segments only incurring standard file system overhead ? This approach is likely to require some VM level changes so unfortunately it is out of my present scope and experience. If there is a good technical reason why it is the best it can ever be, then I'd love to hear it and stand corrected. Otherwise I really hope we stop waving our arms and shruging our shoulders at this each time it is raised. This issue has been in the shadows for years. cheers Denis On 3/8/07, Michael Lucas-Smith <[hidden email]> wrote: > > But is there anything to speed up parcel loading like saving it with > > some special parameters ? > > There's still the stuff I wrote about a few years back where you > disable the system linking as stuff loads. This speeds things up > considerably. Hit the archives searching for parcel loading and you'll > see a few threads talking about this issue. > > Cheers, > Michael > > |
«
Return to VisualWorks
|
1 view|%1 views
| Free forum by Nabble | Edit this page |