Resolving package names...

Hi all..

I decided to rename a package that is part of a bundle that I Store out
to my local Store repository.. Anyway, I renamed it in Store to match
what was in VW's browser listing (per VW's directions).. All was fine..
I then added a new package to my bundle and moved some classes out from
the renamed package to the new package.  Now I find that if I unload my
entire bundle and reload it from my local Store repository, Store
complains that it can't see the base-classes that were moved from
"MyScripUI" to "MyScripUI-Base" and refuses to load anything that refers
to them.. Below is the before & after package layout for my bundle.. Any
ideas on what I've got wrong and how to proceed would be greatly
appreciated..

Before Package Layout :
===========================
MyScrip
   MyScripWidgets (later renamed to MyScripUI in both Store and VW)

After Package Refactoring:
===========================
MyScrip
   MyScripUI-Base (holds base classes)
   MyScripUI (all classes here refer to base classes in above 'Base' pkg)



Below is one of the base class declarations :
===============================================
Smalltalk defineClass: #MSWCommon_Base
    superclass: #{Seaside.WAAsyncComponent}
    indexedType: #none
    private: false
    instanceVariableNames: ''
    classInstanceVariableNames: ''
    imports: '
              Seaside.WARenderCanvas
              Seaside.WAAsyncScripts
             '
    category: 'MyScripUI-Base'



Below is one of the higher class declarations :
===============================================
Smalltalk defineClass: #MSNLI_About
    superclass: #{Smalltalk.MSWCommon_Base}
    indexedType: #none
    private: false
    instancevariableNames: ''
    classInstancevariableNames: ''
    imports: ''
    category: 'MyScripUI'

Rick Flower wrote:
Ok.. I think I know what's causing me grief.. I believe that the newly
relocated base classes are now being loaded AFTER the ones that
reference them.. Is there any easy way to tell Store that the packages
should be ordered in a particular way to ensure dependencies are working
properly?  Thanks!

Also -- Is there any benefit to using a namespace for my application
similar to what Seaside does for all of its stuff?

Rick Flower wrote:

> Ok.. I think I know what's causing me grief.. I believe that the newly
> relocated base classes are now being loaded AFTER the ones that
> reference them.. Is there any easy way to tell Store that the packages
> should be ordered in a particular way to ensure dependencies are working
> properly?  Thanks!

Nevermind.. I found the "edit bundle spec" under the package menu..

One thing we did here that would make sense to have by default is we wired load order validation into publishing so you could never publish a bundle that was out of order. I much rather prefer waiting extra half a second during publish than have to waste a lot of time trying to load the broken bundle later.

Cheers!

-Boris
(Sent from a BlackBerry)

----- Original Message -----
From: Rick Flower <[hidden email]>
To: VisualWorks Mailing List <[hidden email]>
Sent: Fri Mar 30 20:44:19 2007
Subject: Re: Resolving package names...

Rick Flower wrote:

> Ok.. I think I know what's causing me grief.. I believe that the newly
> relocated base classes are now being loaded AFTER the ones that
> reference them.. Is there any easy way to tell Store that the packages
> should be ordered in a particular way to ensure dependencies are working
> properly?  Thanks!

Nevermind.. I found the "edit bundle spec" under the package menu..

Thanks for the suggestion Boris -- I might look into that once my app
starts getting bigger.  For now, I think I'm OK..  I'm certainly
learning more about bundles than I new before and also namespaces.. Very
interesting stuff

-- Rick


Boris Popov wrote:

> One thing we did here that would make sense to have by default is we wired load order validation into publishing so you could never publish a bundle that was out of order. I much rather prefer waiting extra half a second during publish than have to waste a lot of time trying to load the broken bundle later.

Bundle prerequisites and subpackages/subbundle load order should be
eliminated. Package prerequisites already imply a load order, which is
contradicted by the subpackage/subbundle load order of a bundle. And
since bundles themself contain no classes / namespaces / shared
variables, they don't need own prerequisites. The prerequisites of a
bundle are the collected prerequisites of the contained items.

q.e.d. ;)

Disagree with both here. Our application is in a single bundle that contains a number of sub bundles and packages none of which are ever supposed to be loaded on their own, so the simplest method of managing prerequisites is on a bundle level and the load order is defined by package order within bundles, not by creating prerequisite chains between all packages inside. So we couldn't do without both. On the other hand, atomic loading of a bundle (7.5) may eliminate need for explicit load order checking anyway (presumably, I haven't tried it yet) but its not a default yet and thus order validation on publish would serve as a cheap safety measure IMHO.

Cheers!

-Boris
(Sent from a BlackBerry)

----- Original Message -----
From: Holger Kleinsorgen <[hidden email]>
To: [hidden email] <[hidden email]>
Sent: Sat Mar 31 00:45:26 2007
Subject: Re: Resolving package names...


> One thing we did here that would make sense to have by default is we wired load order validation into publishing so you could never publish a bundle that was out of order. I much rather prefer waiting extra half a second during publish than have to waste a lot of time trying to load the broken bundle later.

Bundle prerequisites and subpackages/subbundle load order should be
eliminated. Package prerequisites already imply a load order, which is
contradicted by the subpackage/subbundle load order of a bundle. And
since bundles themself contain no classes / namespaces / shared
variables, they don't need own prerequisites. The prerequisites of a
bundle are the collected prerequisites of the contained items.

q.e.d. ;)

Various actions I could live without, but don't force me to arbitrarily move my prereqs from single top level bundle somewhere down deep arbitrarily. Isn't that what bundles are for? :)

Oh and if you remove actions (which no one will actually do, realistically) only call #loaded on classes after full load completes as those actions likely need to do more than set up that single class.

Cheers!

-Boris
(Sent from a BlackBerry)

----- Original Message -----
From: Travis Griggs <[hidden email]>
To: VW NC <[hidden email]>
Sent: Sat Mar 31 09:49:53 2007
Subject: Re: Resolving package names...

On Mar 31, 2007, at 0:45, Holger Kleinsorgen wrote:



                One thing we did here that would make sense to have by default is we wired load order validation into publishing so you could never publish a bundle that was out of order. I much rather prefer waiting extra half a second during publish than have to waste a lot of time trying to load the broken bundle later.


        Bundle prerequisites and subpackages/subbundle load order should be eliminated. Package prerequisites already imply a load order, which is contradicted by the subpackage/subbundle load order of a bundle. And since bundles themself contain no classes / namespaces / shared variables, they don't need own prerequisites. The prerequisites of a bundle are the collected prerequisites of the contained items.


Amen.

Get rid of load/unload actions, and I might actually like bundles instead of loathing them.

--
Travis Griggs
Objologist
"There are a thousand hacking at the branches of evil to one who is striking at the root" - Henry David Thoreau

You're right, its certainly not the intended use for pundles, but it certainly serves our categorization needs and I suspect we are not the only ones who could care less about what the "right" way is. As to how you can move forward, that would make for an interesting discussion at StS, are you keeping a list? ;) The way I see it, someone built a stadium capable of serving 15 different sports when all you were looking for was a simple hockey rink and people end up wanting to play different games at the same time 24/7 and yet keep screaming for improvements and proper maintenance.

Cheers!

-Boris
(Sent from a BlackBerry)

----- Original Message -----
From: Travis Griggs <[hidden email]>
To: VW NC <[hidden email]>
Sent: Sat Mar 31 10:06:44 2007
Subject: Re: Resolving package names...

On Mar 31, 2007, at 9:42, Boris Popov wrote:

        Disagree with both here. Our application is in a single bundle that contains a number of sub bundles and packages none of which are ever supposed to be loaded on their own, so the simplest method of managing prerequisites is on a bundle level and the load order is defined by package order within bundles, not by creating prerequisite chains between all packages inside. So we couldn't do without both. On the other hand, atomic loading of a bundle (7.5) may eliminate need for explicit load order checking anyway (presumably, I haven't tried it yet) but its not a default yet and thus order validation on publish would serve as a cheap safety measure IMHO.
       
       

So your packages are basically non entities, playing a role akin to "BundleOrganization" (thought it's a pretty crummy version of that since you have categorize along loadable order lines). I won't tell you you shouldn't do that, or to do it another way. It seems to be a common use and was why I prototyped "Fragments" once upon a time (http://www.cincomsmalltalk.com/userblogs/travis/blogView?showComments=true&printTitle=Fragments&entry=3268486869).


I will point out that this "amorphous" model, which everyone wants to use in a different way, and religiously defends as the "the only possible way they could do their work" makes doing anything simple and elegant with them, i.e. tools, almost completely impossible. It hinders our ability to make a nice simple model that is attractive and familiar to new Smalltalk people. It makes tools that do prerequisite computation for example, ugly and difficult to understand. I've worked on tools for a long while now, and even more now, this conviction just grows every time I deal with the bundles and packages and pundles issue.

--
Travis Griggs
Objologist
What's next, Intel Processors branded with "Apple Outside" stickers?

Travis Griggs schrieb am 31.03.2007 19:18:
> Debian/RPM/ports have nothing like bundles. I've heard one proposal that
> bundles are like "directories of directories." Upon recollection, I
> reject this notion. That's like proposing a CVS/SVN where you have a
> project that is a directory, and you can only put files in it, but not
> other directories, and that you can aggregate your "projects" in bigger
> "load it all at once" directories, but that that _those_ directories can
> only contain directories. Programmers would laugh you off the face of
> the planet if you had a model like that.

Including packages in more than one bundle would be what exactly in this
model? A hard link? Or a symbolic link? :-)

I agree that packages and bundles are *not* directories. However, IMO,
it was always a big plus of VisualWorks and its predecessors that it did
*not* have a 1:1 mapping of classes to files (and, as in Java,
applications/packages/whatever to directories). We are working at an
abstraction level which does *not* force us to organize source code in a
structure which can be directly mapped to file system structures.

CVS&Co are systems for versioning *files*, not *source code*. And RPMs
are more like parcels than packages - I don't see them as part of source
code management, but as deployment vehicles. So, I think that it is not
necessary to look for analogies to bundles and packages in this kind of
versioning systems. If we wanted to model our source code management
such that it were compatible to SVN or CVS, we would constrain ourselves
to an inappropriate model IMO.

ENVY was a success because it didn't care whether its model could be
mapped onto files. When it was no longer possible to support it, Cincom
chose Store. Some concepts of Store are similar to ENVY, e.g. bundles
are similar to configuration maps, and packages are similar to
applications. OTOH, there are differences, too. Store is certainly not
perfect, and it was hastily merged into VW without really being
integrated. We have all had fun with categories vs. parcels vs.
packages. I remember a meeting where James Robertson was present, and I
said something like "the object model of Store is a bit lacking". He
answered "What object model?" - which may have been a little bit unfair
to the Store developers. ;-)

Before I get carried away: Store can probably be improved, and there may
be difficulties implementing tools for it. But I certainly don't want to
go back to mimicking file based source code management.

As to load actions: I never liked those, too. Both in ENVY and Store,
they are mere Strings which are not really accessible to the tools. But
you can't just abolish them without replacement.

Blocks which choose which prerequisite versions to load may be necessary
for platform specific configuration in certain environments. Is this
true? Does anybody really use them, or are there other patterns for
solving this kind of problem? Probably, as I have not yet seen a package
with a prereq version selection block in all those years.

Initialization before and after loading / unloading code is certainly
necessary. You won't find anything comparable to this in C - there is no
runtime environment into which the code is loaded. So don't try to find
analogies to this in CVS, there can't be any.

If you are looking for analogies to prerequisite structures, bundles,
and load actions, they might be found in make or Ant, not SVN or CVS.
Loading code into a Smalltalk image is several things at once:
- looking for the necessary components (as in make)
- getting the right versions of those components (as in CVS)
- loading / linking the code
- initializing the runtime environment (which is part of the executable
produced by cc called by make if you program in C).

Where does this lead us? We could live with Store as it is. Or we could
factor out the different aspects of looking up, configuring, versioning,
loading (and unloading), and initializing code. I don't know if this
would leave bundles in the system, but it would certainly introduce some
new concepts to the model. E.g., rules, dependencies, compatibilities,
with prerequisites being mere consequences of these "new" concepts.

Got carried away after all... and found no silver bullet yet. :)

Joachim Geidel

<snip>

I'm not entirely sure where my thoughts and comments fit in this thread,
so I'll just put them here.

Bundles are a poor mans configuration map. But people are also using
them to categorise their classes.

In a more ideal world, like we can put methods in to protocols, we
should be able to put classes in to categories and packages in to
categories. In that respect, Class categories == Fragments and Package
categories == Bundles. In reality, categorisation is a purely human
thing and should have absolutely no impact on load order, prerequisites,
atomicity or anything like what Bundles currently do.

The bundle issue cannot be solved purely by replacing them with
configuration maps, you still need that human-level organisation.

So, in short, I generally agree with Travis, though I think there are
two missing mechanisms - configuration maps and human categorisation.
And to backlink to Joachim, configuration maps is where you find loading
rules and deployment rules.

I sympathise with you Boris, as I used to use bundles to group together
packages that were built purely to aid human categorisation and
therefore had circular dependencies. So I know where you're coming from.
However, bundles aren't really the answer :)

Cheers,
Michael

Michael

With respect to categorization, I was considering providing
for the use of class categories within a Package. On the
browser you would see a + beside a package icon indicating
you can expand it. When expanded one would see the class
categories being used within the package.

A few years ago I worked on a project that used Bundles but
subsequently did away with them. I found them useful for
navigating to packages but deficient as a deployment mechanism.

Terry
 
===========================================================
Terry Raymond       Smalltalk Professional Debug Package
Crafted Smalltalk
80 Lazywood Ln.
Tiverton, RI  02878
(401) 624-4517      [hidden email]
<http://www.craftedsmalltalk.com>
===========================================================

Hi Travis,

> So... can I sum up your argument basically as "since Smalltalk is not
> files, I'm not interested in analogs with other systems?"

I was concerned that drawing analogies between Store and other tools
which have a much more narrow focus may lead to conclusions which ignore
some of the aspects of what bundles are used for in practice.

Michael is right, there are several aspects: categorization, e.g. to
support navigation in tools, and configuration management. Let me add
bundling for deployment to the list, which comes in two forms - loading
code into an image, and building parcel files.

Maybe what we can learn from other solutions is that the different
aspects should be clearly separated, not integrated as in Store. The
StarBrowser seems to address the categorization issue (although I must
admit that I don't know this tool very well), and its classifications
may be a solution for what bundles are used for in many cases.

Joachim

At 05:49 PM 3/31/2007, Travis Griggs wrote:

On Mar 31, 2007, at 12:08, Joachim Geidel wrote:

Travis Griggs schrieb am 31.03.2007 19:18:

Debian/RPM/ports have nothing like bundles. I've heard one proposal that
bundles are like "directories of directories." Upon recollection, I
reject this notion. That's like proposing a CVS/SVN where you have a
project that is a directory, and you can only put files in it, but not
other directories, and that you can aggregate your "projects" in bigger
"load it all at once" directories, but that that _those_ directories can
only contain directories. Programmers would laugh you off the face of
the planet if you had a model like that.

Including packages in more than one bundle would be what exactly in this
model? A hard link? Or a symbolic link? :-)

I agree that packages and bundles are *not* directories. However, IMO,
it was always a big plus of VisualWorks and its predecessors that it did
*not* have a 1:1 mapping of classes to files (and, as in Java,
applications/packages/whatever to directories). We are working at an
abstraction level which does *not* force us to organize source code in a
structure which can be directly mapped to file system structures.

CVS&Co are systems for versioning *files*, not *source code*. And RPMs
are more like parcels than packages - I don't see them as part of source
code management, but as deployment vehicles. So, I think that it is not
necessary to look for analogies to bundles and packages in this kind of
versioning systems. If we wanted to model our source code management
such that it were compatible to SVN or CVS, we would constrain ourselves
to an inappropriate model IMO.

So... can I sum up your argument basically as "since Smalltalk is not files, I'm not interested in analogs with other systems?"

Have you used svn on one or more projects for any length of time? Or put together debian/rpm packages? I'm just curious so I can know from what context to frame a response.

It doesn't seem to me that that's the argument at all.

While analogies to other systems are useful I think one has to be rather careful in trying to draw those analogies. In particular, to draw analogies like "X doesn't have Y, so we don't need Y either". Yes, things like svn/cvs don't have a concept like bundles, as far as I know. They also don't have a concept like packages. Or overrides. Or class extensions. Or classes. Or methods. They're systems for versioning files and directory structures, and the semantics are left entirely to the things that actually use those files.

As far as the analogy to files which would "laughed off the face of the planet", it seems to me that that's also precisely a description of ENVY's configuration maps. And there are quite a few other aspects of Smalltalk development that many users of traditional file-based system do not find compelling. So perhaps a more thorough analysis is in order.

And I'm positive I don't know my own mind on the issue in general;
HOWEVER I do know I have no use for a stored bundle that can't be
loaded! None, zero... does anyone else? If not not can we agree that the
fact that store allows you to do this is a bug? Our fix is quite simple,
run sTORE's own validation prior to storing.

Joerg

Joerg Beekmann wrote:
> And I'm positive I don't know my own mind on the issue in general;
> HOWEVER I do know I have no use for a stored bundle that can't be
> loaded! None, zero... does anyone else? If not not can we agree that the
> fact that store allows you to do this is a bug? Our fix is quite simple,
> run sTORE's own validation prior to storing.

In my case, my manual validation was to store the bundle in question
into my Store repository, save my image, then unload the bundle and
try to re-load it from the repository.. If it didn't work, then exit the
image and start up again and fix the unresolved issue.. I did that
several times and it eventually ferreted out all of the problems.