On 24.08.2010, at 01:44, Levente Uzonyi wrote:
> On Tue, 24 Aug 2010, Bert Freudenberg wrote: > >> On 24.08.2010, at 00:34, Levente Uzonyi wrote: >> >>> On Tue, 24 Aug 2010, Bert Freudenberg wrote: >>> >>>> On 24.08.2010, at 00:03, Levente Uzonyi wrote: >>>> >>>>> The problem with updating after unloading is that the updater mechanism has no information of package dependencies, so it will try to load packages randomly which obviously won't work. >>>> >>>> It does not load packages randomly, but in the order defined in the update config map. >>> >>> That's right, but it's still random from the package dependency POV. >> >> What makes you think so? The order is chosen (manually) so that dependent packages are loaded before depending packages. > > If that would be the case, then updating an "unloaded" image would reload all unloaded packages. Indeed. That's how it is supposed to work. > But that doesn't work. For example 311Deprecated depends on Traits, but Traits is at the end of the list while 311Deprecated is the first non-dummy package. Then, obviously, the update map is wrong :) - Bert - |
On 8/23/2010 4:53 PM, Bert Freudenberg wrote:
> On 24.08.2010, at 01:44, Levente Uzonyi wrote: >> But that doesn't work. For example 311Deprecated depends on Traits, but Traits is at the end of the list while 311Deprecated is the first non-dummy package. > > Then, obviously, the update map is wrong :) No, it's not. The update map isn't ordered to allow reloading all the packages after they've been unloaded. The update is ordered to allow loading all the updates *after the previous update map*. In fact, it can and does happen that it switches the ordering of two packages between updates. For example, consider moving class Process from Kernel to System. To do this you would need the System package to be loaded before the Kernel. But if you would load these from first principles, almost certainly you would load Kernel before System. So the order in the update map is fairly irrelevant for the order which you would use to load all these packages from first principles. The update map simply cannot be used for this purpose. The real question here is if we should allow updating a subset of packages simply by skipping over the packages you haven't present in your image? This could be done but would come at the price that we could no longer add new "required" packages and let people fetch them automatically (because they wouldn't be present in the image they would not be loaded). In addition, allowing to update subsets would require us to be *much* more careful with changing package dependencies - as you can see from the failing package dependency tests in the current trunk the dependencies *do* change but if we want to allow updating subsets then we have to be "dependency clean" in every single update. The latter is really the reason why I'm not in favor of allowing this. Right now we have a solution that basically says when you update incrementally, you update an image with all the packages present which keeps it nice and simple. At the release points, we spend the additional time to make sure all the dependencies are in the right place and we can unload things but we don't have to worry about accidentally introducing a dependency in the middle and getting angry messages from people whose system we broke since they only update a subset of the packages. Makes sense? If not, then let's discuss ways in which we can improve things. Cheers, - Andreas |
On 24.08.2010, at 05:32, Andreas Raab wrote:
> On 8/23/2010 4:53 PM, Bert Freudenberg wrote: >> On 24.08.2010, at 01:44, Levente Uzonyi wrote: >>> But that doesn't work. For example 311Deprecated depends on Traits, but Traits is at the end of the list while 311Deprecated is the first non-dummy package. >> >> Then, obviously, the update map is wrong :) > > No, it's not. The update map isn't ordered to allow reloading all the packages after they've been unloaded. The update is ordered to allow loading all the updates *after the previous update map*. In fact, it can and does happen that it switches the ordering of two packages between updates. For example, consider moving class Process from Kernel to System. To do this you would need the System package to be loaded before the Kernel. But if you would load these from first principles, almost certainly you would load Kernel before System. > > So the order in the update map is fairly irrelevant for the order which you would use to load all these packages from first principles. The update map simply cannot be used for this purpose. > > The real question here is if we should allow updating a subset of packages simply by skipping over the packages you haven't present in your image? This could be done but would come at the price that we could no longer add new "required" packages and let people fetch them automatically (because they wouldn't be present in the image they would not be loaded). I was thinking to add a specific list of known unloaded packages. Only those would be skipped when updating. > In addition, allowing to update subsets would require us to be *much* more careful with changing package dependencies - as you can see from the failing package dependency tests in the current trunk the dependencies *do* change but if we want to allow updating subsets then we have to be "dependency clean" in every single update. True. I'm just saying that by declaring a package unloadable, we assert it is dependency-clean. > The latter is really the reason why I'm not in favor of allowing this. Right now we have a solution that basically says when you update incrementally, you update an image with all the packages present which keeps it nice and simple. At the release points, we spend the additional time to make sure all the dependencies are in the right place and we can unload things but we don't have to worry about accidentally introducing a dependency in the middle and getting angry messages from people whose system we broke since they only update a subset of the packages. > > Makes sense? If not, then let's discuss ways in which we can improve things. > > Cheers, > - Andreas I agree, but my point was slightly different ;) Manually re-ordering the update map is just a work-around for not being able to update multiple packages together. If we could, the order would not matter for moving classes/methods around. But that shouldn't affect the unloadable packages because, by definition, there are no dependencies from the core packages to them. So they could go to the end of the update list, and be ordered in a way to allow reloading. What I'm saying is that unless one is trying to move things between the unloadable packages, I see no reason to not have them in loadable order. - Bert - |
On 8/24/10, Bert Freudenberg <[hidden email]> wrote:
> What I'm saying is that unless one is trying to move things between the > unloadable packages, I see no reason to not have them in loadable order. You mean that you want to treat all the unloaded packages as one large image segment? The attached drawing is intended to illustrate that we are talking about Trunk update (black arrow , the current package) and update of the image with packages unloaded (green arrow). And it seems that 'Reloading' has to be addresses as well (Red arrow). The black and the green Trunk update could be combined in a single update which distinguishes what packages are there (that was one suggestion in this thread). The other idea is just to have two different trunk updates - one for the minimal image, one for the unloaded packages. I read in the meeting notes that the infrastructure has to be set up for this and as the discussion in this thread has show so far 1) Unloading works actually fine 2) But there is the need for an infrastructure around it (this includes 'reloading') --Hannes MinimalTrunkImage.png (17K) Download Attachment |
On 24.08.2010, at 19:55, Hannes Hirzel wrote:
> On 8/24/10, Bert Freudenberg <[hidden email]> wrote: >> What I'm saying is that unless one is trying to move things between the >> unloadable packages, I see no reason to not have them in loadable order. > > You mean that you want to treat all the unloaded packages as one large > image segment? No. I was just talking about the update map. "Package" means "source code" here. - Bert - > The attached drawing is intended to illustrate that we are talking > about Trunk update (black arrow , the current package) and update of > the image with packages unloaded (green arrow). And it seems that > 'Reloading' has to be addresses as well (Red arrow). > > The black and the green Trunk update could be combined in a single > update which distinguishes what packages are there (that was one > suggestion in this thread). > The other idea is just to have two different trunk updates - one for > the minimal image, one for the unloaded packages. > > I read in the meeting notes that the infrastructure has to be set up > for this and as the discussion in this thread has show so far > > 1) Unloading works actually fine > 2) But there is the need for an infrastructure around it (this > includes 'reloading') > > --Hannes > <MinimalTrunkImage.png> |
In reply to this post by Andreas.Raab
On Mon, Aug 23, 2010 at 8:32 PM, Andreas Raab <[hidden email]> wrote:
I always appreciated the 95% rule, a.k.a. don't let the perfect be the enemy of the good. Right now if one removes all packages and updates one gets an ugly mess one can wade through with the debugger. If one were to simply skip the packages not in one's image one could certainly get to a point that might miss a required package, but one could also update without wading through the mess, without having to go through the remove unloadable packages step and without having to go through whatever other steps one had gone through to produce a finished image.
E.g. take my OS Cog VMMaker image which has been derived from an unload step to be nice to svn users; the image is half the size of the normal one. It's a mild pain to build, loading packages, adding workspaces and repositories etc. Yes I could automate all this but it would be much nicer to build it once and periodically update it. I don't care if it misses some required package; I can deal with that issue when I get to it. But not being able to update without wading through mud is, to put it mildly, a downer.
|
On 8/24/2010 2:12 PM, Eliot Miranda wrote:
> I always appreciated the 95% rule, a.k.a. don't let the perfect be the > enemy of the good. Right now if one removes all packages and updates > one gets an ugly mess one can wade through with the debugger. If one > were to simply skip the packages not in one's image one could certainly > get to a point that might miss a required package, but one could also > update without wading through the mess, without having to go through the > remove unloadable packages step and without having to go through > whatever other steps one had gone through to produce a finished image. > > E.g. take my OS Cog VMMaker image which has been derived from an unload > step to be nice to svn users; the image is half the size of the normal > one. It's a mild pain to build, loading packages, adding workspaces and > repositories etc. Yes I could automate all this but it would be much > nicer to build it once and periodically update it. I don't care if it > misses some required package; I can deal with that issue when I get to > it. But not being able to update without wading through mud is, to put > it mildly, a downer. I don't think you understand what I mean by "required" package :-) Consider that we're splitting the Morphic package along the lines that Pavel proposed. We add a MorphicWidgets package that includes all the Pluggable*Morphs. The problem is that the next version of the Morphic package will show these as REMOVALS so if you update the Morphic package and leave out the MorphicWidgets your image is an instant goner. That's what I mean by "required"; not some random goodie that nobody cares if it's absent. Worse, this wouldn't just happen to people who have deliberately chosen to update only a subset; it would happen to *everyone* updating their trunk images and make it pretty much impossible to perform such operations. Unless we can find a solution to this problem, I think Bert's alternative is the only reasonable one - add an explicit list of ignored packages for the updater and you can add to that list whatever you want and it won't get updated. You'll have to deal with the consequences if something goes wrong there but as you said, you can deal with that when you get to it. Cheers, - Andreas |
On Tue, Aug 24, 2010 at 3:04 PM, Andreas Raab <[hidden email]> wrote:
No, I get it, and it'll crash horribly. But it crashes horribly _now_. I don't mind rebuilding and unloading when I have to, e.g. because package structure changes force it. But I sure as sugar hate having to do it when it's not forced.
|
In reply to this post by Andreas.Raab
On Tue, Aug 24, 2010 at 3:04 PM, Andreas Raab <[hidden email]> wrote:
BTW, I'm not proposing relaxing this rule for trunk images, only for images in which packages have been unloaded, even packages in which all unloadable packages have been unloaded. I don't want to wreck updating, all I want is for updating to work to some acceptable degree in unload-all-unloadable-packages (UAUP) images.
I'm sorry, I thought I was being clear and wasn't. It is IMO preferrable to add a hack for UAUP images that allows them to update the packages they have available than the current situation where updating is possible only in a full trunk image.
best Eliot
|
On 8/24/2010 6:53 PM, Eliot Miranda wrote:
> BTW, I'm not proposing relaxing this rule for trunk images, only for > images in which packages have been unloaded, even packages in which all > unloadable packages have been unloaded. I don't want to wreck updating, > all I want is for updating to work to some acceptable degree in > unload-all-unloadable-packages (UAUP) images. > > I'm sorry, I thought I was being clear and wasn't. It is IMO > preferrable to add a hack for UAUP images that allows them to update the > packages they have available than the current situation where updating > is possible only in a full trunk image. Give it a shot; just load the latest version of MonticelloConfigurations into the image you want to update, disable the "Update missing packages" preference, press update, and let me know if it blows up or not :-) (the change is minor and trivial to retract if it turns out to be more trouble than it's worth) Cheers, - Andreas |
On Tue, Aug 24, 2010 at 8:10 PM, Andreas Raab <[hidden email]> wrote:
Excellent; works perfectly! The preference must be updated manually because there's no preferences browser (for others evaluate (MCMcmUpdater classPool at: #UpdateMissingPackages put: false)) but with that done update went without a hitch. Thank you!
|
On 8/24/2010 9:03 PM, Eliot Miranda wrote:
> Excellent; works perfectly! The preference must be updated manually > because there's no preferences browser (for others evaluate > (MCMcmUpdater classPool at: #UpdateMissingPackages put: false)) but with > that done update went without a hitch. Thank you! No, no, no. This is why we have those tagged preferences. All you need to do is to evaluate: MCMcmUpdater updateMissingPackages: false. No need to poke in the classPool or anything :-) Cheers, - Andreas |
Eliot and Andreas,
Are you saying that the following works perfectly? 1) Take http://ftp.squeak.org/trunk/Squeak4.2-10382-alpha.zip 2) Evaluate SmalltalkImage unloadAllKnownPackages. MCMcmUpdater updateMissingPackages: false. 3) Choose 'mouse' menu, 'Update Squeak' --Hannes On 8/25/10, Andreas Raab <[hidden email]> wrote: > On 8/24/2010 9:03 PM, Eliot Miranda wrote: >> Excellent; works perfectly! The preference must be updated manually >> because there's no preferences browser (for others evaluate >> (MCMcmUpdater classPool at: #UpdateMissingPackages put: false)) but with >> that done update went without a hitch. Thank you! > > No, no, no. This is why we have those tagged preferences. All you need > to do is to evaluate: > > MCMcmUpdater updateMissingPackages: false. > > No need to poke in the classPool or anything :-) > > Cheers, > - Andreas > > |
In reply to this post by Andreas.Raab
I'm really happy about this. Some things were broken in the unloaded image (world menu items that depended on missing pieces) but other things were more exigent at the time, and the long time to unload discouraged me from testing much in the unloaded image.
Speaking of which, does it make sense to break the tests up into a group that's still relevant to the unloaded image? If it does, and we do that, I wouldn't mind running the unload and then running the tests the next time I put up an updated image; I can put up one full image as well as its unloaded counterpart. Perhaps that would stir more interest in hacking on the core image. I suppose that might not work, though: seems like committing from an unloaded image would probably cause methods to be removed. Drat. On Aug 24, 2010, at 9:06 PM, Andreas Raab <[hidden email]> wrote: > On 8/24/2010 9:03 PM, Eliot Miranda wrote: >> Excellent; works perfectly! The preference must be updated manually >> because there's no preferences browser (for others evaluate >> (MCMcmUpdater classPool at: #UpdateMissingPackages put: false)) but with >> that done update went without a hitch. Thank you! > > No, no, no. This is why we have those tagged preferences. All you need to do is to evaluate: > > MCMcmUpdater updateMissingPackages: false. > > No need to poke in the classPool or anything :-) > > Cheers, > - Andreas > |
This has piqued my interest, so I've committed a couple packages to
the inbox that should: * Automatically upon unloading a package, it will no longer try to reload this when updating from trunk * When manually reloading the package, updating from trunk will once again try to update the package That seems to me a fuller answer to this thread. -Chris |
Great idea. Thank you for acting quickly. This brings us forward in an
important area. If I am not mistaken this has made it into the trunk by now. Is this correct? Retest needed then. There is an issue about naming the global variable which keeps references to packages which are not supposed to be updated by the trunk update mechanism (Bert Freudenberg, I second it). --Hannes On 8/25/10, Chris Cunningham <[hidden email]> wrote: > This has piqued my interest, so I've committed a couple packages to > the inbox that should: > * Automatically upon unloading a package, it will no longer try to > reload this when updating from trunk > * When manually reloading the package, updating from trunk will once > again try to update the package > > That seems to me a fuller answer to this thread. > > -Chris > > |
In reply to this post by Andreas.Raab
On Tue, Aug 24, 2010 at 9:06 PM, Andreas Raab <[hidden email]> wrote:
That's what comes from reading the commit comment; the class variable shows up first ;)
|
In reply to this post by Hannes Hirzel
On Tue, Aug 24, 2010 at 10:03 PM, Hannes Hirzel <[hidden email]> wrote: Eliot and Andreas, I haven't tried starting from Squeak 4.2-xxxxx. I started from a 4.1 image, changed the update URL to trunk, did SmalltalkImage unloadAllKnownPackages, loaded the various Cog VMMaker packages, loaded the latest MonticelloConfigurations, did the moral equivalent of MCMcmUpdater updateMissingPackages: false. and then step 3. But I would expect that the above would work provided that you insert the extra step:
1.5) Make sure the latest MonticelloConfigurations is loaded 2) Evaluate SmalltalkImage unloadAllKnownPackages. MCMcmUpdater updateMissingPackages: false. 3) Choose 'mouse' menu, 'Update Squeak'
|
In reply to this post by Hannes Hirzel
On Wed, Aug 25, 2010 at 3:56 AM, Hannes Hirzel <[hidden email]> wrote:
> If I am not mistaken this has made it into the trunk by now. Is this > correct? Retest needed then. No, this isn't in trunk yet. Yes, it needs testing. > There is an issue about naming the global variable which keeps > references to packages which are not supposed to be updated by the > trunk update mechanism (Bert Freudenberg, I second it). The Class Variable (not globlal) has been renamed to be, well, more obvious about what it's intent is. -Chris |
Oh, and for anyone trying this, you will need to load the
MonticelloConfigurations before Monticello changes. |
Free forum by Nabble | Edit this page |