Name spaces in Spoon

On 5/26/06, Michael Latta <[hidden email]> wrote:

> I can see the need to have methods that are defined by different authors
> that happen to use the same selector, but are really different methods.  As
> the size of libraries grows this is more and more likely.  I suspect that
> method collision is far more likely and more of an issue than class name
> collision.  Possibly what is needed is method name prefixes more than class
> name prefixes?

I've been programming in Smalltalk off and on for twenty years.  I've
taught a class that used it at least once a year.  Method collision
that could be solved by private selectors is pretty rare.  Class
collision is more common, but still isn't that common.

I've seen three types of method collisions.  One is where people
extend a class like Collection with new methods that extend the notion
of what a collection is.  These methods are not based on the users of
the class, but are intrinsic to the class.  When two modules try to
add methods with the same name, they usually do more or less the same
thing.  If they don't do the same thing, it is easy for one module to
rename its version of the method.  It is not common, but private
selectors could prevent it.

Another potential type of method collision is where a module needs
some default behavior in all classes and so adds a method to base
classes like Object.  These methods really have little to do with
Object and are only going to be sent by the module,.  Often their name
is based on the module, and so the chance of collision is vrery small.
 I don't think I've ever seen this kind of collision, though I often
make this kind of extension of existing classes.

The third type of method collision is where a module changes an
existing framework by overriding some of its methods.  This is the
most common kind of method collision, but private selctors wouldn't
help, because the whole point is to use the same selectors that the
framework is already using.

-Ralph Johnson

See below.  Outlook does not support normal quoting.

On 5/26/06, Michael Latta <[hidden email]> wrote:

> I can see the need to have methods that are defined by different authors
> that happen to use the same selector, but are really different methods.
As
> the size of libraries grows this is more and more likely.  I suspect that
> method collision is far more likely and more of an issue than class name
> collision.  Possibly what is needed is method name prefixes more than
class
> name prefixes?

I've been programming in Smalltalk off and on for twenty years.  I've
taught a class that used it at least once a year.  Method collision
that could be solved by private selectors is pretty rare.  Class
collision is more common, but still isn't that common.

I've seen three types of method collisions.  One is where people
extend a class like Collection with new methods that extend the notion
of what a collection is.  These methods are not based on the users of
the class, but are intrinsic to the class.  When two modules try to
add methods with the same name, they usually do more or less the same
thing.  If they don't do the same thing, it is easy for one module to
rename its version of the method.  It is not common, but private
selectors could prevent it.

#MAL I think one of the ideas is that you should be able to load a
#MAL module without modification.  In particular if the idea is to host
#MAL multiple applications in one image, no modification should be required.

Another potential type of method collision is where a module needs
some default behavior in all classes and so adds a method to base
classes like Object.  These methods really have little to do with
Object and are only going to be sent by the module,.  Often their name
is based on the module, and so the chance of collision is vrery small.
 I don't think I've ever seen this kind of collision, though I often
make this kind of extension of existing classes.

#MAL While having the module name in the selector reduces collisions
#MAL it also means that the selector does not read as well as regular
#MAL selectors.

The third type of method collision is where a module changes an
existing framework by overriding some of its methods.  This is the
most common kind of method collision, but private selctors wouldn't
help, because the whole point is to use the same selectors that the
framework is already using.

#MAL If you mean replacing a method in one module by a modified method
#MAL in another module that has a very different semantic.  That is more
#MAL a matter of having the loader be able to do that.  The methods do not
#MAL co-exist.  If you mean to
#MAL implement a selector defined in another module, that will surely be
#MAL required in any solution.  Not all methods can be private, just as a
#MAL class in a module would need to be able to override a method in the
#MAL base class of any classes it defines.

-Ralph Johnson

On May 26, 2006, at 8:26 PM, Andreas Raab wrote:

Yes! An excellent approach.

I think we rely on globals way, way too much. It makes some things  
simpler, but it comes with a high price.

Colin

I'm sure I'm dreaming but, to me, a class should be distinguished from
another class with the same name by a compressed answer to the
questions, who, what, where, when, why, & how.

One can then filter which classes, with whatever name, can be active
at a given time based on filtering on any of the above criteria. E.g.
filter out all modifications of a class by such-and-such person or
company (known to produce buggy code for example). Filter to only
activate classes created before a certain date. Or a date range with a
gap in it. Or that solve a particular problem listed/named in the
"why" section. Or exclude classes written in a certain "where" region.
Or filter on the "how" of the "licensing" the author gave the code. Or
filter out classes targeted for "who" - being children (which might
exclude things of interest to the child-like such as myself :-) ).

The "when" & "who" in combination would make each class unique.

This way, creation of "modules" can be more "automatic" or "in the
background" and not draw a lot on the programmer's attention, which,
when a burden on one's attention, becomes and incentive to ignore the
technology.

I'm sure others have thought of this strategy, but I've not heard of
one, so I don't have the name for it.

Cheers,
Darius

While I'm still dreaming ... I also have a problem with method names
that use words which can have more than one meaning, in English or any
other language. What's the "point"? Seems to me that a word in a
method name should be able to "link" to its (more unique, thought not
perfect) semantic & multi-lingual meaning and not guessed from the
context, class use, module name, etc.

Words in method names used as idioms can also "throw" non-native
English speakers for a "loop".

The "why" can also be a scientific discipline which uses the concepts
modeled by the class.

Cheers,
Darius

P.S. by "Or filter on the "how" of the "licensing" the author gave the
code." I meant ... the "how" of the distribution license chosen by the
author of the code that the code is release under by that author. ...

> From: Michael Latta
> The downside I see in having call context dependent behavior
> is that the
> reader of the code needs to know the run-time context that
> will modify the
> behavior of the object.  The same result can be achieved with
> a wrapper
> object that implements the new methods, and passes on unknown
> methods to the delegate.

Consider the following: ClassA implements methodA as '^self'.  ClassB, a
subclass of A, implements methodB as '^self methodA'.  Now a module adds
methodA to ClassB (let's say it's '^self') and calls 'ClassB new
methodB'.  The delegate case would call ClassA>>methodA, whereas the
'expected' behaviour is pobably a call of the module's methodA.  There's
also an interesting discussion to be had about what class the returned
object would be - it could be ClassB or ClassB-delegate depending on how
this was implemented

This is not 'the same result', I think.

                - Peter

On 5/26/06, [hidden email] <[hidden email]> wrote:
What i like with java is classloaders. Java does not know where the
classes are. Instead each class has an classloader and java uses that
classloader to get a handle to the code for a classname. At
compile-time (actually load-time). So that classloader is kind of an
active namespace.
How about doing something similar for a starter? The compiler passes
names to  an user-hook and the hook passes back such a handle? Or an
"expanded" classname.
One way to use that is, use very long classnames globally,
MyGlobalContextHasThisClass, inside code use MThisClass and let that
hook translate.
Allow some more syntax, so people who like MyGobalContext::ThisClass
could do it. Write some real code in your favorite style. Better base
for discussions.

That translator could be stored by categories, a category in the
browser would show its naming-rules.

For refactoring there could be the other way around too. Then for
refactor-tools a method is first expanded to long names, refactored,
and abbreviated by the categories rules. Current tools still work.

> regards, Göran
>
>


--
-Volker

"Any problem in computer science can be solved with another layer of
indirection. But that usually will create another problem." David
Wheeler

On 5/26/06, Michael Latta <[hidden email]> wrote:
>
> #MAL I think one of the ideas is that you should be able to load a
> #MAL module without modification.  In particular if the idea is to host
> #MAL multiple applications in one image, no modification should be required.

One solution I've seen (I think that Envy did this, but it has been a
few years, so perhaps I am not remembering correctly) is for your
module system to complain if you try to load two modules that want to
change the same method, and the modules are independent of each other.
 If one module is a prerequisite of the other then the latter module
is assumed to know what it is doing when it changes the first.  But if
the modules are independent and they try to change the same method
then they will not load until you tell the system what to do about the
conflict. It is sort of like a merge conflict in a configuration
management system.

In practice, ths is not a problem, because people don't generate
selector names randomly.  For example, Morphic adds  #isMorph and
#costumes to Object.  It is possible that someone else might add
#costumes to Object, but nobody would try to define #isMorph unless
they were trying to cause trouble.

I am not sure what you are saying.  If you are saying "a private
selector system will not solve this problem" then that is the point I
was trying to make.

In my opinion, the first two kinds of problems are easy to fix without
private selectors, and the third problem will not be helped by private
selectors, so I fail to see why private selectors are useful.  I think
this is another case of people getting excited about solving a hard
problem, but it is better to just ignore the problem because it is not
important.

-Ralph Johnson

> From: Ralph Johnson
> people don't generate
> selector names randomly.  For example, Morphic adds  #isMorph and
> #costumes to Object.  It is possible that someone else might add
> #costumes to Object, but nobody would try to define #isMorph unless
> they were trying to cause trouble.

Consider an effort to construct a simpler Morphic (or refactor it).  You
might want exactly that behaviour during development of the new Morphic,
where both the old and the new versions defined the same method on the
same class with different implementations.  You almost certainly *would*
want it in a putative future image where MVC had gone, and hence the
only toolset you could use to construct the new Morphic was running
using features of the old Morphic.

Spoon+Naiad could resolve this in either of two ways, as I undertand it.
One way would be to run the "new" Morphic in its own object memory and
remote-debug it into existence; this is independent of namespaces.  The
other would be to allow both in the same image at the same time; this is
hightly dependent on namespaces.  Craig will no doubt tel me I'm talking
through an inappropriate orifice here...

If we're planning on refactoring Squeak, we're going to come across
instances like this.  At present, one has to be very, very careful when
changing some parts of the image, as it's very easy to break something
and leave oneself without access to tools - this is the responsibility
that comes with the power of being able to change anything 'live'.  I'd
love to see a way of reducing this danger, and I'm very interested that
Craig appears to have provided us with two distinct ways - one of which
we appear to be intent on removing from consideration!

                - Peter

This is sort of my point exactly.  The object's behavior is different
depending on the sender of the message.  While some systems claim that to be
a valuable "feature" (I think Slate does that), I think it will just cause
confusion.  The argument for this often goes along the lines of supporting
different roles for an object in different contexts.  Joe as a Father at
home behaves differently than Joe as a Manager at work.  With the right tool
support it might seem natural to always ask "what context am I in when
reading this code?", but I think the cure is worse than the disease.  If
someone has actually used this type of system I would be interested in their
experience.

Michael

I think this is exactly why the approach of using two images is better.
With Spoon I can change the target image as needed to get it from the "old"
state to the "new" state without it needing to be functional in support of
tools or even a UI.  If you tried to refactor Morphic in-place the risk to
functioning UI would be too great.  I see no need for a module system to
attempt to address this problem, as the Spoon approach will always be far
better.

A good argument can be made to always use two images.  Image 1 is always
tools, and image 2 is always domain objects.

Michael

Le Samedi 27 Mai 2006 04:43, Ralph Johnson a écrit :
> I've been programming in Smalltalk off and on for twenty years.  I've
> taught a class that used it at least once a year.  Method collision
> that could be solved by private selectors is pretty rare.  Class
> collision is more common, but still isn't that common.

However, name clashes are probable when loading two packages dealing with same
subject.

For example
    Number>>,
to form an array of numbers or a matrix, or:
    SequenceableCollection>>asMatrix

If i load two Matrix packages in my image, they will likely override these
selectors.

Why to load two Matrix packages ? Maybe it's not my choice, but the dependency
chain of end user apps i was interested in...

Same for two File/URL packages, two HTML rendering packages etc...

Nicolas

I forgot another case from end user point of view: it can happen that two
packages A & B depend on two different versions of a package C...

Bad case for names clashes (all updated methods indeed).

Nicolas

Le Samedi 27 Mai 2006 23:01, nicolas cellier a écrit :

Hi andreas

in fact I like the idea that a module only refer to its own names and  
that these names can point to other names by contrast with an  
approach where all code of the modules can refer to foreign names.

This is what we tried to have in classboxes with import statement.  
After the notion of parametrization
is based on where the import is specified. If this is outside a  
module and you can "duplicate code" then you get effective  
parametrization. Else you get modules that you can parametrized but  
you cannot get two at the same
time in memory

I once asked Dan if having import statement at the modules boundaries  
would help the image-segment
analysis (since you would not have to go through all the code) just  
the local name base of the module you want to save. But dan never  
replied :)


Stef

Hi folks!

First a few comments in no particular order:

- I have decided to again go through my Namespaces code and will publish
and do a boringly detailed description of how it works, method by
method. :)

- Note that the :: syntax *is* easy to implement - I can say that
because Andreas helped me do it in the Namespaces code which is
available on SM and *works fine*. QED.

- If we go for a runtime messages approach as Dan describes then another
disadvantage is the analysis of code. Currently I am working on some
analysis code that is intended to run in the SM server that figures out
dependencies between packages based on their references and definitions
of globals. This would not really work if we move this to runtime, or at
least it gets definitely more complicated.

Andreas wrote:
Let me mention that Henrik actually had stuff along these lines
(parametrization just like above) in modules 3.3a.

But considering these issues in light of my Namespaces solution it also
seems very easy to "remap" bindings at compile time - because the
references are *always* qualified fully in the source. So for example,
before filing in code that uses Foo::X, we could remap Foo:: to Bar::
and thus have Foo::X resolved to Bar::X. (note that it is pretty nice to
be able to say "Foo::" instead of "the namespace called 'Foo'" or
"namespace Foo")

And another note: My code does not forbid good ole globals. In fact, I
probably propose that we do NOT split up the current "basic" image into
any namespaces at all. If you file in my Namespaces code into an image
you can then proceed working *just as you do now*. There are NO
differences. Well, ok, some small tiny differences are there at the
moment - but I intend to turn those "off" using Preferences with default
values false. So soon there will be NO differences.

I also agree with Michael Rueger (as Dan mentioned) that hierarchies
*are* complex. Especially considering if we start making remappings like
the above possible. I urge us all to "remember the Alamo" - 3.3a
modules. It was too foreign, too "complicated", it made experienced
Smalltalkers fumble and lose the ball. And the result was that noone
moved into the "New House Of Modularity" - and thus it died. Please, do
not underestimate the problems with such "advanced" approaches -
especially if they radically change the "taste and feel" of Smalltalk.


Now... to move ahead a bit. Modules...


A simple proposal for Modules
-----------------------------------

I strongly claim that Namespaces and Modules are two different things -
even though they in practice often (other languages) are handled in the
same mechanism since they mostly overlap. Someone said once on
squeak-dev (one of the Alans on the list IIRC) that the only interesting
"meaning" of a module is "separately deployable unit". I very much agree
with that.

So then, if we consider a Module to be a unit of code (let's consider
only code for the moment - it doesn't hamper the idea AFAICT) then we
have:

1. References to "globals" that are not in the module itself.
2. Defined "globals" in the module.

This is like inputs and outputs. :) Of course - I am using the term
"globals" so that we can relate this to how things work today. Also note
that these are named objects - not necessarily classes - but again, let
us focus on code.

Now - if we install (=file in, create classes, compile code) a module A
it will have to resolve all foreign globals (#1) above and it will add
its own globals (#2). Monticello and regular Squeak deals with missing
globals in various ways - if I am not mistaken the Association is
created and put in Undeclared. This means that all compiled methods
referring to say nonexistent class Banana are linked to the same
Association - but with a value of nil.

And then if you install or create class Banana it will "reuse" that
Association in Undeclared (and remove it from Undeclared). So this is
how Squeak generally deals with installing code without having to worry
too much about load order. All references to Banana will refer to the
same object - regardless if they were compiled before or after Banana
was created.

I want to make this mechanism visible and much more tangible. I want to
be able to install a "module" A into the image without having it be
"activated" (just like Henrik had in 3.3a). Then I can look at it and
see that yes, it still is not functional because its binding with the
class Banana is still not fulfilled. Then I can install module B (which
has Banana) and see that module A is now fulfilled and should work if we
activate it together with module B.

So to summarize:

1. I want to use my Namespaces code, which more or less is "prefixes
done right". This introduces class Namespace and each prefix corresponds
to an instance of Namespace held in Smalltalk. Or expressed in
Smalltalk:

        Foo::X == ((Smalltalk at: #Foo) at: #X)
        Foo == (Smalltalk at: #Foo)
        Foo class == Namespace

2. A simple module system would be to create class Module and let each
instance represent a "piece" of the image with inputs and outputs like
above. It could have behaviors like activate, deactivate, hasAllInputs,
inputs, outputs and so on. How a Module defines its boundaries can be
discussed - perhaps it has a list of PIs, or whatever. And again, it can
have a few different implementations depending on if it is just a chunk
of object memory with named inputs/outputs or if we actually know it is
code.

I am not sure if I made the above easily understandable nor if it is a
good way to do it. But IMHO it seems simple, *understandable* and rather
generic. It is all about hooking a piece of object memory into the image
and make sure it can reconnect its connections in both directions
(inputs and outputs) using name lookup.

regards, Göran

> Goran wrote:
>
> - If we go for a runtime messages approach as Dan describes then another
> disadvantage is the analysis of code. Currently I am working on some
> analysis code that is intended to run in the SM server that figures out
> dependencies between packages based on their references and definitions
> of globals. This would not really work if we move this to runtime, or at
> least it gets definitely more complicated.

I would think this would make it easier.  The globals become the modules
themselves and the messages just access classes/globals from those modules.


>
> I also agree with Michael Rueger (as Dan mentioned) that hierarchies
> *are* complex. Especially considering if we start making remappings like
> the above possible. I urge us all to "remember the Alamo" - 3.3a
> modules. It was too foreign, too "complicated", it made experienced
> Smalltalkers fumble and lose the ball. And the result was that noone
> moved into the "New House Of Modularity" - and thus it died. Please, do
> not underestimate the problems with such "advanced" approaches -
> especially if they radically change the "taste and feel" of Smalltalk.

I agree with this one.  I had not used VisualWorks for many years, and
looking at the current image is quite a shock.  The hierarchical namespaces
and imports is much more complex than required.

> I strongly claim that Namespaces and Modules are two different things -

I strongly agree.  Modules is the ability to package up code/data/objects so
that they can be manipulated as a unit, loaded and unloaded reliably, and
not interfere with the operation of each other.

> So then, if we consider a Module to be a unit of code (let's consider
> only code for the moment - it doesn't hamper the idea AFAICT) then we
> have:

I would rather say as in all of Smalltalk that a module is a set of objects.
Many of the objects in a module will represent executable code.  But, I see
no reason to prevent data objects.  For example if I deploy a module that
includes an LALR parser, I will want to deploy the LALR parse table and
such.  If I deploy a tax module I would want to deploy the tax tables.  One
of the big strengths of Smalltalk is that I do not have to represent
everything in code, I can have objects in the image that support the code,
and I do not need to recreate them each time the "application" starts up.

I like this approach.  The bindings are explicit in each module for all
external reference, and get resolved at load time.  The bindings from the
module get activated separately to allow random module load order.  You
could even have methods on the Module object that perform many operations
like canActivate and exportBindings, etc.

So in your system we can have a single module contributing to multiple
namespaces, and namespaces that span modules.  That seems appropriate, the
tricky part will be that both modules will need to ensure that the Namespace
object exists but neither can own it.  So the load process of a module can
create an object that is outside the module.

Ideally each module could be an image segment and loaded as a block with one
memory mapped I/O.  That may be a bit ambitious in the first phase, but
seems a desirable end goal.

It might be simpler to just have Module == Namespace.  While I can see the
value in the two part approach it also adds complications of its own that
will be rarely used.  I think it far more likely that a module will want to
attach a method to a class owned by another module, and that is not
addressed in this proposal as it is not just a name binding issue.  To do
that properly there needs to be a "partial" class in the module that is
merged into the real class at load time, and can be unloaded as well, or
each Class must have an "extensions" collection that holds all such
"partial" classes and are treated as extensions.  While more flexible and
cleaner this too is fairly complex and not to be taken on until a proven
need is seen.  I would go with inserting the methods on load into the
existing class, but having a "partial" class object in the module that is
responsible for the load/unload process.

>
> I am not sure if I made the above easily understandable nor if it is a
> good way to do it. But IMHO it seems simple, *understandable* and rather
> generic. It is all about hooking a piece of object memory into the image
> and make sure it can reconnect its connections in both directions
> (inputs and outputs) using name lookup.
>
> regards, Göran

Michael Latta

> In fact, I probably propose that we do NOT split up the current  
> "basic" image into
> any namespaces at all.

But conceptually you do it. This is not because you do not touch  
Smalltalk that you do not do it.
And this is crying for that.

Stef

Hi Peter--

 > One way would be to run the "new" Morphic in its own object memory and
 > remote-debug it into existence; this is independent of namespaces. The
 > other would be to allow both in the same image at the same time; this
 > is highly dependent on namespaces.  Craig will no doubt tell me I'm
 > talking through an inappropriate orifice here...

        Hee. :)  Actually, in the second case you could install the core of
Naiad, which among other things allows every class to have an arbitrary
name (and create modules, and transfer them elsewhere). I don't think
namespaces would ever be necessary in either case.


-C

--
Craig Latta
improvisational musical informaticist
www.netjam.org
Smalltalkers do: [:it | All with: Class, (And love: it)]