Towards SqueakCore
Locked
123
123
 
|
In the interests of revisiting Pavel Krivanek's work, and a long term
goal of this community, I thought I'd use the Dependency Browser and dig out interpackage dependencies. By scraping the DependencyBrowser's contents together with a bit of UI scripting I've constructed a dotfile of Trunk (attached). Turning this into a PNG results in an 11MB image! [1] Nodes near the top are nodes that aren't used by many things. For instance, ReleaseBuilder's right at the top because nothing depends on it. One thing to note is that XML-Parser and Nebraska are only used by Universes, and that Universes isn't used by anything else. It occurs to me that we could thus remove these 3 packages from trunk and add the loading of these to ReleaseBuilderFor4dot5 [2], and still end up with a 4.5 that while apparently unchanged, actually has a smaller core. What do you think of trying this out as an experiment? How would we unload these packages? (I should note: I've nothing against these packages. They're just packages that aren't woven into the guts of the image, and are thus easily removable.) frank [1] If you have dot installed, `dot -Tpng -o trunk-deps.dot trunk-deps.png` will do the trick. [2] Installer squeak package: 'trunk'; install: '39Deprecated-ar.19'; install: '311Deprecated-nice.2'; install: 'XML-Parser-ael.35'; install: 'Nebraska-ul.35'; install: 'Universes-nice.45'. |
trunk-deps.zip (3K) |
|
Doesn't sound like a bad idea. One concern though, might be to see how different e.g. XML-Parser is in trunk from whichever canonical repo hangs onto it. Push things that may have changed with it in the trunk upstream, then unload it.
Getting to a smaller core system is worth doing!
On Sat, Feb 9, 2013 at 3:51 PM, Frank Shearar <[hidden email]> wrote: In the interests of revisiting Pavel Krivanek's work, and a long term Casey Ransberger |
|
|
In reply to this post by Frank Shearar-3
On Sat, Feb 09, 2013 at 11:51:21PM +0000, Frank Shearar wrote:
> In the interests of revisiting Pavel Krivanek's work, and a long term > goal of this community, I thought I'd use the Dependency Browser and > dig out interpackage dependencies. > > By scraping the DependencyBrowser's contents together with a bit of UI > scripting I've constructed a dotfile of Trunk (attached). Turning this > into a PNG results in an 11MB image! [1] Nodes near the top are nodes > that aren't used by many things. > > For instance, ReleaseBuilder's right at the top because nothing depends on it. > > One thing to note is that XML-Parser and Nebraska are only used by > Universes, and that Universes isn't used by anything else. > > It occurs to me that we could thus remove these 3 packages from trunk > and add the loading of these to ReleaseBuilderFor4dot5 [2], and still > end up with a 4.5 that while apparently unchanged, actually has a > smaller core. > > What do you think of trying this out as an experiment? How would we > unload these packages? (I should note: I've nothing against these > packages. They're just packages that aren't woven into the guts of the > image, and are thus easily removable.) I would prefer to see the experiment focus on *reloadable* packages, in the sense of SmalltalkImage>>unloadAllKnownPackages, where the unloaded packages are supposedly distinct enough that they can be reloaded after having been removed from the image. Success would be defined as being able to unload a package, reload it, and verify that the image is equivalent to what you started with. I think that the packages you identified are probably good candidates for an initial experiment with this. If we had some way to actually test that reloadable packages stay that way over time, presumably with the help of Jenkins, it would be a really good thing. I'm not entirely sure how to test for this, but it would be very worthwhile if someone could figure it out. I think that a verifiable set of truly reloadable packages would go a long way toward improving modularity. I'm not so keen on the idea of just unloading things without some mechanism to verify that they stay reloadable. That's a fast track to bit rot. > > frank > > [1] If you have dot installed, `dot -Tpng -o trunk-deps.dot > trunk-deps.png` will do the trick. I think you meant 'dot -Tpng -o trunk-deps.png trunk-deps.dot'. Nice picture :) > [2] Installer squeak > package: 'trunk'; > install: '39Deprecated-ar.19'; > install: '311Deprecated-nice.2'; > install: 'XML-Parser-ael.35'; > install: 'Nebraska-ul.35'; > install: 'Universes-nice.45'. > |
|
|
On 2/9/13 9:43 PM, "David T. Lewis" <[hidden email]> wrote: > On Sat, Feb 09, 2013 at 11:51:21PM +0000, Frank Shearar wrote: >> In the interests of revisiting Pavel Krivanek's work, and a long term >> goal of this community, I thought I'd use the Dependency Browser and >> dig out interpackage dependencies. >> >> By scraping the DependencyBrowser's contents together with a bit of UI >> scripting I've constructed a dotfile of Trunk (attached). Turning this >> into a PNG results in an 11MB image! [1] Nodes near the top are nodes >> that aren't used by many things. >> >> For instance, ReleaseBuilder's right at the top because nothing depends on >> it. >> >> One thing to note is that XML-Parser and Nebraska are only used by >> Universes, and that Universes isn't used by anything else. >> >> It occurs to me that we could thus remove these 3 packages from trunk >> and add the loading of these to ReleaseBuilderFor4dot5 [2], and still >> end up with a 4.5 that while apparently unchanged, actually has a >> smaller core. >> >> What do you think of trying this out as an experiment? How would we >> unload these packages? (I should note: I've nothing against these >> packages. They're just packages that aren't woven into the guts of the >> image, and are thus easily removable.) > > I would prefer to see the experiment focus on *reloadable* packages, > in the sense of SmalltalkImage>>unloadAllKnownPackages, where the > unloaded packages are supposedly distinct enough that they can be > reloaded after having been removed from the image. Success would > be defined as being able to unload a package, reload it, and verify > that the image is equivalent to what you started with. I think that > the packages you identified are probably good candidates for an initial > experiment with this. |
|
|
In reply to this post by David T. Lewis
On 2/10/13, David T. Lewis <[hidden email]> wrote:
> On Sat, Feb 09, 2013 at 11:51:21PM +0000, Frank Shearar wrote: >> In the interests of revisiting Pavel Krivanek's work, and a long term >> goal of this community, I thought I'd use the Dependency Browser and >> dig out interpackage dependencies. >> >> By scraping the DependencyBrowser's contents together with a bit of UI >> scripting I've constructed a dotfile of Trunk (attached). Turning this >> into a PNG results in an 11MB image! [1] Nodes near the top are nodes >> that aren't used by many things. >> >> For instance, ReleaseBuilder's right at the top because nothing depends on >> it. >> >> One thing to note is that XML-Parser and Nebraska are only used by >> Universes, and that Universes isn't used by anything else. >> >> It occurs to me that we could thus remove these 3 packages from trunk >> and add the loading of these to ReleaseBuilderFor4dot5 [2], and still >> end up with a 4.5 that while apparently unchanged, actually has a >> smaller core. >> >> What do you think of trying this out as an experiment? How would we >> unload these packages? (I should note: I've nothing against these >> packages. They're just packages that aren't woven into the guts of the >> image, and are thus easily removable.) > > I would prefer to see the experiment focus on *reloadable* packages, > in the sense of SmalltalkImage>>unloadAllKnownPackages, where the > unloaded packages are supposedly distinct enough that they can be > reloaded after having been removed from the image. Success would > be defined as being able to unload a package, reload it, and verify > that the image is equivalent to what you started with. I think that > the packages you identified are probably good candidates for an initial > experiment with this. Yes. > > If we had some way to actually test that reloadable packages stay > that way over time, presumably with the help of Jenkins, it would be > a really good thing. Yes, the question is how do we define success (i.e. construct a test) for a use case like 1) take the full image 2) unload a package 3) reload the package 4) check if we the full image is still OK I'm not entirely sure how to test for this, but > it would be very worthwhile if someone could figure it out. I think > that a verifiable set of truly reloadable packages would go a long > way toward improving modularity. Yes, > I'm not so keen on the idea of just unloading things without some > mechanism to verify that they stay reloadable. Sure, I understand that Frank wants the same thing. He indentified XML-Parser and Nebraska as candidate for unloadable packages. That's a fast track > to bit rot. I think the way out is to keep the current full image and construct tests for unloading and reloading which can run on Jenkins. --Hannes >> >> frank >> >> [1] If you have dot installed, `dot -Tpng -o trunk-deps.dot >> trunk-deps.png` will do the trick. > > I think you meant 'dot -Tpng -o trunk-deps.png trunk-deps.dot'. > Nice picture :) > > >> [2] Installer squeak >> package: 'trunk'; >> install: '39Deprecated-ar.19'; >> install: '311Deprecated-nice.2'; >> install: 'XML-Parser-ael.35'; >> install: 'Nebraska-ul.35'; >> install: 'Universes-nice.45'. > > >> > > > |
|
|
In reply to this post by Frank Shearar-3
Hi,
I recommend to follow the same way we use in Pharo: - firstly prepare the SqueakCore and clean it a bit. - make basic packages like Network and Monticello loadable. - make the rest of the system loadable using Monticello at once, ensure everything is working well - start to improve granularity of that bundle Unfortunately, I have to say that Squeak is eons back from the point of modularity. Cheers, -- Pavel On Sun, Feb 10, 2013 at 12:51 AM, Frank Shearar <[hidden email]> wrote: > In the interests of revisiting Pavel Krivanek's work, and a long term > goal of this community, I thought I'd use the Dependency Browser and > dig out interpackage dependencies. > > By scraping the DependencyBrowser's contents together with a bit of UI > scripting I've constructed a dotfile of Trunk (attached). Turning this > into a PNG results in an 11MB image! [1] Nodes near the top are nodes > that aren't used by many things. > > For instance, ReleaseBuilder's right at the top because nothing depends on it. > > One thing to note is that XML-Parser and Nebraska are only used by > Universes, and that Universes isn't used by anything else. > > It occurs to me that we could thus remove these 3 packages from trunk > and add the loading of these to ReleaseBuilderFor4dot5 [2], and still > end up with a 4.5 that while apparently unchanged, actually has a > smaller core. > > What do you think of trying this out as an experiment? How would we > unload these packages? (I should note: I've nothing against these > packages. They're just packages that aren't woven into the guts of the > image, and are thus easily removable.) > > frank > > [1] If you have dot installed, `dot -Tpng -o trunk-deps.dot > trunk-deps.png` will do the trick. > [2] Installer squeak > package: 'trunk'; > install: '39Deprecated-ar.19'; > install: '311Deprecated-nice.2'; > install: 'XML-Parser-ael.35'; > install: 'Nebraska-ul.35'; > install: 'Universes-nice.45'. > > > |
|
|
Hello Pavel
Thank you for your recommendation. As not having followed closely what you and others did in Pharo, may I ask you to give a short summary where you have reached now with Pharo core effort. I have seen quite a number of smaller Pharo images on the Pharo integration server but I am not sure what the status of them is. https://ci.inria.fr/pharo/view/Pharo-Kernel-2.0/ Please let me ask very basic questions. 1. Are they all derived from the the main current "full" Pharo2.0? 2. What is the quality of them? 3. How are they used by people? Thank you in advance --Hannes On 2/11/13, Pavel Krivanek <[hidden email]> wrote: > Hi, > > I recommend to follow the same way we use in Pharo: > - firstly prepare the SqueakCore and clean it a bit. > - make basic packages like Network and Monticello loadable. > - make the rest of the system loadable using Monticello at once, > ensure everything is working well > - start to improve granularity of that bundle > > Unfortunately, I have to say that Squeak is eons back from the point > of modularity. > > Cheers, > -- Pavel > > On Sun, Feb 10, 2013 at 12:51 AM, Frank Shearar <[hidden email]> > wrote: >> In the interests of revisiting Pavel Krivanek's work, and a long term >> goal of this community, I thought I'd use the Dependency Browser and >> dig out interpackage dependencies. >> >> By scraping the DependencyBrowser's contents together with a bit of UI >> scripting I've constructed a dotfile of Trunk (attached). Turning this >> into a PNG results in an 11MB image! [1] Nodes near the top are nodes >> that aren't used by many things. >> >> For instance, ReleaseBuilder's right at the top because nothing depends on >> it. >> >> One thing to note is that XML-Parser and Nebraska are only used by >> Universes, and that Universes isn't used by anything else. >> >> It occurs to me that we could thus remove these 3 packages from trunk >> and add the loading of these to ReleaseBuilderFor4dot5 [2], and still >> end up with a 4.5 that while apparently unchanged, actually has a >> smaller core. >> >> What do you think of trying this out as an experiment? How would we >> unload these packages? (I should note: I've nothing against these >> packages. They're just packages that aren't woven into the guts of the >> image, and are thus easily removable.) >> >> frank >> >> [1] If you have dot installed, `dot -Tpng -o trunk-deps.dot >> trunk-deps.png` will do the trick. >> [2] Installer squeak >> package: 'trunk'; >> install: '39Deprecated-ar.19'; >> install: '311Deprecated-nice.2'; >> install: 'XML-Parser-ael.35'; >> install: 'Nebraska-ul.35'; >> install: 'Universes-nice.45'. >> >> >> > > |
|
|
In reply to this post by Pavel Krivanek
> I recommend to follow the same way we use in Pharo:
> - firstly prepare the SqueakCore and clean it a bit. > - make basic packages like Network and Monticello loadable. > - make the rest of the system loadable using Monticello at once, > ensure everything is working well > - start to improve granularity of that bundle If I understand correctly, this would mean starting from another (smaller) image than the current trunk, and try to get to the trunk by adding packages. It seems to me that backward compatibility is jeopardized in this process (I know it's not a Pharo value, but it is a Squeak one). I prefer the process of starting from the trunk and having more and more packages unloadable/loadable (in that order). This way the trunk is always there and backward compatibility can be verified at any time. Stef |
|
|
In reply to this post by Hannes Hirzel
On Mon, Feb 11, 2013 at 1:05 PM, H. Hirzel <[hidden email]> wrote:
> Hello Pavel > > Thank you for your recommendation. As not having followed closely what > you and others did in Pharo, may I ask you to give a short summary > where you have reached now with Pharo core effort. > > I have seen quite a number of smaller Pharo images on the Pharo > integration server but I am not sure what the status of them is. > > https://ci.inria.fr/pharo/view/Pharo-Kernel-2.0/ > > Please let me ask very basic questions. > > 1. Are they all derived from the the main current "full" Pharo2.0? Yes, they are, the generation of this images is based on set of scripts that we modify to reflect latests changes in Pharo. > 2. What is the quality of them? For the Pharo Kernel we have 3 Undeclared, no obsolete classes, 117 unimplemented calls and 7 failing tests. There is still a lot of space for improvements. Others are more dirty. > 3. How are they used by people? I do not know about any real-life usage except usages by me. But we used them a lot for testing and improving of projects like Fuel/Tanker and bootstrapping. So we was able to produce full Morphic image generated by bootstrapping. Because we have a CI setting, we can see if some update has bad effect on modularity and UI independency of Pharo. Primary goal is not to have minimal system but to have clean and modular system. Cheers, -- Pavel > > Thank you in advance > > --Hannes > > On 2/11/13, Pavel Krivanek <[hidden email]> wrote: >> Hi, >> >> I recommend to follow the same way we use in Pharo: >> - firstly prepare the SqueakCore and clean it a bit. >> - make basic packages like Network and Monticello loadable. >> - make the rest of the system loadable using Monticello at once, >> ensure everything is working well >> - start to improve granularity of that bundle >> >> Unfortunately, I have to say that Squeak is eons back from the point >> of modularity. >> >> Cheers, >> -- Pavel >> >> On Sun, Feb 10, 2013 at 12:51 AM, Frank Shearar <[hidden email]> >> wrote: >>> In the interests of revisiting Pavel Krivanek's work, and a long term >>> goal of this community, I thought I'd use the Dependency Browser and >>> dig out interpackage dependencies. >>> >>> By scraping the DependencyBrowser's contents together with a bit of UI >>> scripting I've constructed a dotfile of Trunk (attached). Turning this >>> into a PNG results in an 11MB image! [1] Nodes near the top are nodes >>> that aren't used by many things. >>> >>> For instance, ReleaseBuilder's right at the top because nothing depends on >>> it. >>> >>> One thing to note is that XML-Parser and Nebraska are only used by >>> Universes, and that Universes isn't used by anything else. >>> >>> It occurs to me that we could thus remove these 3 packages from trunk >>> and add the loading of these to ReleaseBuilderFor4dot5 [2], and still >>> end up with a 4.5 that while apparently unchanged, actually has a >>> smaller core. >>> >>> What do you think of trying this out as an experiment? How would we >>> unload these packages? (I should note: I've nothing against these >>> packages. They're just packages that aren't woven into the guts of the >>> image, and are thus easily removable.) >>> >>> frank >>> >>> [1] If you have dot installed, `dot -Tpng -o trunk-deps.dot >>> trunk-deps.png` will do the trick. >>> [2] Installer squeak >>> package: 'trunk'; >>> install: '39Deprecated-ar.19'; >>> install: '311Deprecated-nice.2'; >>> install: 'XML-Parser-ael.35'; >>> install: 'Nebraska-ul.35'; >>> install: 'Universes-nice.45'. >>> >>> >>> >> >> > |
|
|
In reply to this post by Stéphane Rollandin
On Mon, Feb 11, 2013 at 3:18 PM, Stéphane Rollandin
<[hidden email]> wrote: >> I recommend to follow the same way we use in Pharo: >> - firstly prepare the SqueakCore and clean it a bit. >> - make basic packages like Network and Monticello loadable. >> - make the rest of the system loadable using Monticello at once, >> ensure everything is working well >> - start to improve granularity of that bundle > > > If I understand correctly, this would mean starting from another (smaller) > image than the current trunk, and try to get to the trunk by adding > packages. It seems to me that backward compatibility is jeopardized in this > process (I know it's not a Pharo value, but it is a Squeak one). > > I prefer the process of starting from the trunk and having more and more > packages unloadable/loadable (in that order). This way the trunk is always > there and backward compatibility can be verified at any time. > > Stef Hi Stef, it does not directly mean to start from another image because the smaller image would be generated from the trunk image for every update. It is only a way how to ensure that the packages are loadable back - only a more radical version of "unloadAllKnownPackages". It is a parallel process of checking the image quality and modularity. The way "from top" is very hard if you must deal with big dirty packages like Morphic and co. Cheers, -- Pavel |
|
|
In reply to this post by Stéphane Rollandin
On 11/02/13 9:18 AM, Stéphane Rollandin wrote:
> ... It seems to me that backward compatibility is > jeopardized in this process (I know it's not a Pharo value, but it is a > Squeak one). <rant> What's up with some people in the Squeak "camp" trying to artificially draw distinctions between Squeak and Pharo - to the point of FUD, like the above statement. Of course "backward compatibility" is of value to everybody. A less antagonistic way to state the Pharo point of view, is that: if backward compatibility would block forward progress, or is too costly to provide, then, choose to drop compatibility (and if proven wrong in that decision, put back the compatibility - which has occurred already). How is that different from what Squeak has to do? IMHO, the difference is in the cost the community is willing to pay for compatibility, and the rate at which forward progress can be made. That's the choice individuals and communities choose, nothing wrong with that. </rant> |
|
|
>> ... It seems to me that backward compatibility is
>> jeopardized in this process (I know it's not a Pharo value, but it is a >> Squeak one). > > <rant> > What's up with some people in the Squeak "camp" trying to artificially > draw distinctions between Squeak and Pharo - to the point of FUD, like > the above statement. Keep cool, man. I just expressed a fact, and a worry as far as I am concerned, because I do need backward compatibility if I expect muO to grow with Squeak. I have no problem with Pharo project nor people, so please don't create trouble where there is none. This is a technical discussion. Stef |
|
|
In reply to this post by Yanni Chiu
Yanni,
I think Stéphane refers to the original Pharo manifesto which clearly states "no backward compatibility". http://code.google.com/p/pharo/ However the current Pharo web page has a mission statement http://www.pharo-project.org/about sets a much more moderate tone. In any case in this thread we want to move on towards a Squeak core and learn from the Pharo experience as much as possible. Please let us not digress from this important topic. Maybe we should follow both at the same time Let me call it - the Pavel Krivanek approach and - the SmalltalkImage unloadAllKnownPackages approach BTW #unloadAllKnownPackages used to work in Squeak 4.1, see http://lists.squeakfoundation.org/pipermail/squeak-dev/2010-August/152427.html So there is no reason why we should not manage to get it working again in Squeak 4.5alpha. And Pavel's approach may be followed in parallel. Because fixing one thing will help the other and vice-verse. --Hannes On 2/11/13, Yanni Chiu <[hidden email]> wrote: > On 11/02/13 9:18 AM, Stéphane Rollandin wrote: >> ... It seems to me that backward compatibility is >> jeopardized in this process (I know it's not a Pharo value, but it is a >> Squeak one). > > <rant> > What's up with some people in the Squeak "camp" trying to artificially > draw distinctions between Squeak and Pharo - to the point of FUD, like > the above statement. > > Of course "backward compatibility" is of value to everybody. A less > antagonistic way to state the Pharo point of view, is that: if backward > compatibility would block forward progress, or is too costly to provide, > then, choose to drop compatibility (and if proven wrong in that > decision, put back the compatibility - which has occurred already). > > How is that different from what Squeak has to do? IMHO, the difference > is in the cost the community is willing to pay for compatibility, and > the rate at which forward progress can be made. That's the choice > individuals and communities choose, nothing wrong with that. > </rant> > > > |
|
|
In reply to this post by David T. Lewis
On 10 February 2013 00:43, David T. Lewis <[hidden email]> wrote:
> On Sat, Feb 09, 2013 at 11:51:21PM +0000, Frank Shearar wrote: >> In the interests of revisiting Pavel Krivanek's work, and a long term >> goal of this community, I thought I'd use the Dependency Browser and >> dig out interpackage dependencies. >> >> By scraping the DependencyBrowser's contents together with a bit of UI >> scripting I've constructed a dotfile of Trunk (attached). Turning this >> into a PNG results in an 11MB image! [1] Nodes near the top are nodes >> that aren't used by many things. >> >> For instance, ReleaseBuilder's right at the top because nothing depends on it. >> >> One thing to note is that XML-Parser and Nebraska are only used by >> Universes, and that Universes isn't used by anything else. >> >> It occurs to me that we could thus remove these 3 packages from trunk >> and add the loading of these to ReleaseBuilderFor4dot5 [2], and still >> end up with a 4.5 that while apparently unchanged, actually has a >> smaller core. >> >> What do you think of trying this out as an experiment? How would we >> unload these packages? (I should note: I've nothing against these >> packages. They're just packages that aren't woven into the guts of the >> image, and are thus easily removable.) > > I would prefer to see the experiment focus on *reloadable* packages, > in the sense of SmalltalkImage>>unloadAllKnownPackages, where the > unloaded packages are supposedly distinct enough that they can be > reloaded after having been removed from the image. Success would > be defined as being able to unload a package, reload it, and verify > that the image is equivalent to what you started with. I think that > the packages you identified are probably good candidates for an initial > experiment with this. > > If we had some way to actually test that reloadable packages stay > that way over time, presumably with the help of Jenkins, it would be > a really good thing. I'm not entirely sure how to test for this, but > it would be very worthwhile if someone could figure it out. I think > that a verifiable set of truly reloadable packages would go a long > way toward improving modularity. > > I'm not so keen on the idea of just unloading things without some > mechanism to verify that they stay reloadable. That's a fast track > to bit rot. I don't see how you're disagreeing with me. I realised after my initial post that the packages I'd taken as examples are actually unloadable, in the sense that they're in the list in #unloadAllKnownPackages. What I'm suggesting is the inverse of #unloadAllKnownPackages: strip these out of Trunk, and add them in as part of preparing a release. The only difference to the current Trunk then - in theory - is that we'd want to take a ReleaseTrunk artifact and run all its tests (because the SqueakTrunk job obviously wouldn't run the tests). Or, we expand the ExternalPackage-Foo jobs to cover all the packages added by the ReleaseBuilder. (One problem: Installer's marked as being unloadable, and it's probably one of the last packages we'd want to unload. The ExternalPackage-Foo scripts I'm writing use Installer. But then, I could always have those scripts load Installer as a prerequisite.) frank >> frank >> >> [1] If you have dot installed, `dot -Tpng -o trunk-deps.dot >> trunk-deps.png` will do the trick. > > I think you meant 'dot -Tpng -o trunk-deps.png trunk-deps.dot'. > Nice picture :) > > >> [2] Installer squeak >> package: 'trunk'; >> install: '39Deprecated-ar.19'; >> install: '311Deprecated-nice.2'; >> install: 'XML-Parser-ael.35'; >> install: 'Nebraska-ul.35'; >> install: 'Universes-nice.45'. > > >> > > |
|
|
In reply to this post by Frank Shearar-3
[Re-send because of message size] Dear all,Am 10.02.2013 um 00:51 schrieb Frank Shearar <[hidden email]>: In the interests of revisiting Pavel Krivanek's work, and a long term So I played around with the data a bit. I ran the dot file through tred (part of graphviz), to eliminate dependencies that are already satisfied through transitive relations. (deps-simplified) It turns out we have many cycles, and tred complaints about: cycle involves edge Files -> System I then used cluster (also part of graphviz) to identify clusters in the dependency graph (found three around Kernel, Collections, and Tests) (deps-simplified-reclustered) and manually added a "morphic" and a "HelpSystem" cluster (deps-simplified-reclustered-manucluster) Note the big circle from System over ST80 to Morphic and from there via Ballon/Collections back to System, or via Monticello/Kernel/Collection back to System. Apart from that, note a direct circl (MonticelloMocks<->Tests) but also a small indirect circles (System->Environments->Compiler->System).  If we aim for modularity, we should invest time to investigate those circles. Best -Tobias |
|
|
In reply to this post by Frank Shearar-3
On 2/12/13, Frank Shearar <[hidden email]> wrote:
> (One problem: Installer's marked as being unloadable, and it's > probably one of the last packages we'd want to unload. Yes, the Installer package was probably included to demonstrate that it is indeed unloadable. The > ExternalPackage-Foo scripts I'm writing use Installer. But then, I > could always have those scripts load Installer as a prerequisite.) > For what is the aim now. Unloading packages and building a release by reloading them again on the integration server the package 'Installer' should be excluded from the list. Are there other candidates which should stay? --Hannes code snippet taken from SmalltalkImage>>unloadAllKnownPackages "Go unloading" #( 'ReleaseBuilder' 'ScriptLoader' '311Deprecated' '39Deprecated' 'Universes' 'SMLoader' 'SMBase' 'Installer-Core' 'VersionNumberTests' 'VersionNumber' 'Services-Base' 'PreferenceBrowser' 'Nebraska' 'ToolBuilder-MVC' 'ST80' 'CollectionsTests' 'GraphicsTests' 'KernelTests' 'MorphicTests' 'MultilingualTests' 'NetworkTests' 'ToolsTests' 'TraitsTests' 'SystemChangeNotification-Tests' 'FlexibleVocabularies' 'EToys' 'Protocols' 'XML-Parser' 'Tests' 'SUnitGUI' 'Help-Squeak' 'HelpSystem' 'SystemReporter' ) do: [:pkgName| (MCPackage named: pkgName) unload. MCMcmUpdater disableUpdatesOfPackage: pkgName. ]. |
|
|
In reply to this post by Tobias Pape
On 12 February 2013 13:54, Tobias Pape <[hidden email]> wrote:
> > [Re-send because of message size] > Dear all, > > Am 10.02.2013 um 00:51 schrieb Frank Shearar <[hidden email]>: > > > In the interests of revisiting Pavel Krivanek's work, and a long term > goal of this community, I thought I'd use the Dependency Browser and > dig out interpackage dependencies. > > By scraping the DependencyBrowser's contents together with a bit of UI > scripting I've constructed a dotfile of Trunk (attached). Turning this > into a PNG results in an 11MB image! [1] Nodes near the top are nodes > that aren't used by many things. > > > > So I played around with the data a bit. Wow, thanks Tobias! Great stuff! > I ran the dot file through tred (part of graphviz), to eliminate dependencies that are > already satisfied through transitive relations. (deps-simplified) > It turns out we have many cycles, and tred complaints about: > cycle involves edge Files -> System > > I then used cluster (also part of graphviz) to identify clusters in the > dependency graph (found three around Kernel, Collections, and Tests) > (deps-simplified-reclustered) and manually added a "morphic" and a "HelpSystem" > cluster (deps-simplified-reclustered-manucluster) > > Note the big circle from System over ST80 to Morphic and from there via Ballon/Collections > back to System, or via Monticello/Kernel/Collection back to System. That's one circle we'd want to lose. I think we can break that just by moving MailComposition >> #addAttachment to Tools (thus breaking Network's usage of Tools). (There might be a bunch of things to do, but that's definitely a dependency we want to break.) > Apart from that, note a direct circl (MonticelloMocks<->Tests) but also a small > indirect circles (System->Environments->Compiler->System). > > > > If we aim for modularity, we should invest time to investigate those circles. Some of those circles we may never resolve. Craig Latta might have some insight here, because he had to deal with these issues in the creation of his micro-images. Craig? frank > Best > -Tobias > [Find the formerly attached zip at http://netshed.de/trunk-deps.zip ] > > > > |
|
|
In reply to this post by Frank Shearar-3
On Tue, Feb 12, 2013 at 01:40:21PM +0000, Frank Shearar wrote:
> On 10 February 2013 00:43, David T. Lewis <[hidden email]> wrote: > > On Sat, Feb 09, 2013 at 11:51:21PM +0000, Frank Shearar wrote: > >> In the interests of revisiting Pavel Krivanek's work, and a long term > >> goal of this community, I thought I'd use the Dependency Browser and > >> dig out interpackage dependencies. > >> > >> By scraping the DependencyBrowser's contents together with a bit of UI > >> scripting I've constructed a dotfile of Trunk (attached). Turning this > >> into a PNG results in an 11MB image! [1] Nodes near the top are nodes > >> that aren't used by many things. > >> > >> For instance, ReleaseBuilder's right at the top because nothing depends on it. > >> > >> One thing to note is that XML-Parser and Nebraska are only used by > >> Universes, and that Universes isn't used by anything else. > >> > >> It occurs to me that we could thus remove these 3 packages from trunk > >> and add the loading of these to ReleaseBuilderFor4dot5 [2], and still > >> end up with a 4.5 that while apparently unchanged, actually has a > >> smaller core. > >> > >> What do you think of trying this out as an experiment? How would we > >> unload these packages? (I should note: I've nothing against these > >> packages. They're just packages that aren't woven into the guts of the > >> image, and are thus easily removable.) > > > > I would prefer to see the experiment focus on *reloadable* packages, > > in the sense of SmalltalkImage>>unloadAllKnownPackages, where the > > unloaded packages are supposedly distinct enough that they can be > > reloaded after having been removed from the image. Success would > > be defined as being able to unload a package, reload it, and verify > > that the image is equivalent to what you started with. I think that > > the packages you identified are probably good candidates for an initial > > experiment with this. > > > > If we had some way to actually test that reloadable packages stay > > that way over time, presumably with the help of Jenkins, it would be > > a really good thing. I'm not entirely sure how to test for this, but > > it would be very worthwhile if someone could figure it out. I think > > that a verifiable set of truly reloadable packages would go a long > > way toward improving modularity. > > > > I'm not so keen on the idea of just unloading things without some > > mechanism to verify that they stay reloadable. That's a fast track > > to bit rot. > > I don't see how you're disagreeing with me. I realised after my > initial post that the packages I'd taken as examples are actually > unloadable, in the sense that they're in the list in > #unloadAllKnownPackages. I'm not disagreeing with you :) Dave > > What I'm suggesting is the inverse of #unloadAllKnownPackages: strip > these out of Trunk, and add them in as part of preparing a release. > The only difference to the current Trunk then - in theory - is that > we'd want to take a ReleaseTrunk artifact and run all its tests > (because the SqueakTrunk job obviously wouldn't run the tests). > > Or, we expand the ExternalPackage-Foo jobs to cover all the packages > added by the ReleaseBuilder. > > (One problem: Installer's marked as being unloadable, and it's > probably one of the last packages we'd want to unload. The > ExternalPackage-Foo scripts I'm writing use Installer. But then, I > could always have those scripts load Installer as a prerequisite.) > > frank > > >> frank > >> > >> [1] If you have dot installed, `dot -Tpng -o trunk-deps.dot > >> trunk-deps.png` will do the trick. > > > > I think you meant 'dot -Tpng -o trunk-deps.png trunk-deps.dot'. > > Nice picture :) > > > > > >> [2] Installer squeak > >> package: 'trunk'; > >> install: '39Deprecated-ar.19'; > >> install: '311Deprecated-nice.2'; > >> install: 'XML-Parser-ael.35'; > >> install: 'Nebraska-ul.35'; > >> install: 'Universes-nice.45'. > > > > > >> > > > > |
|
|
I've never yet been convinced that *un*loading packages as a general technique is a good idea; unload them once to make the small core image and make the packages *load* nicely. Being able to unload seems likely to require tracking what was changed so you can restore it - and not just once but potentially many times with odd combinations for multi layers of loaded packages. I'm not sure it is practical to have a system that avoids any possibility of interactions but I'd certainly be happy to be wrong.
My suspicion is that we will need some very good tools to help make, maintain, test and care for loadable packages. It's so easy for things to slide into cruftulescense. tim -- tim Rowledge; [hidden email]; http://www.rowledge.org/tim "How many tnuctip does it take to change a lightbulb?" "Depends what you want them to change it into." |
|
|
In reply to this post by Tobias Pape
On Tue, Feb 12, 2013 at 5:54 AM, Tobias Pape <[hidden email]> wrote:
Yes! We should especially look at System->GetText One thing that seems odd is that many packages depend on Tests, but very few depend on SUnit. You would think that all those TestCase subclasses would create dependencies on SUnit, not Tests.
Sorting out all the test packages might be an easy place to start. Colin |
«
Return to Squeak - Dev
|
1 view|%1 views
| Free forum by Nabble | Edit this page |