Yet another namespace proposal (was Re: Quick comparison of two Namespaces proposals)

Ok, I've stated my reasons why I don't like Göran's approach, now time
to give a suggestion myself:

What if namespaces were just objects that behave like dictionaries
with the key being a string and value being a meta-class?

They could be used as in:

Seaside at: #Component

or perhaps with a special shorthand:

Seaside Component   "maybe Component needs a #?"

This way it is just an object like everything else, with full
reflection and everything else we love about Smalltalk.  The only
issue is that this is a pure runtime lookup.  We would probably like
to have at least the option of it being compile time.

No problem, we just need my other previous proposal:  Macros. :)  Then
you can make the namespace lookup at compile time with:

##(Seaside at: #Component)

or:

##(Seaside Component)

So what do you all think? :)

P.S.  Oh yea, I didn't say anything about imports etc.  Well, I
haven't thought about that part yet. :)  Maybe a later email.  :)

> ##(Seaside at: #Component)
>
> or:
>
> ##(Seaside Component)
>
> So what do you all think? :)

Would you like to type it or prefer Seaside.Component (or
Seaside::Component, you get the idea)?  Not from a philosophic POV, but
rather from a very pragmatic and aesthetic POV.

Paolo

--
http://people.squeakfoundation.org/person/mikevdg
http://gulik.pbwiki.com/

Hi all!

As I have hinted on IRC etc I am not at all sure my own proposal from a
few years back is "optimal" - even though I tend to defend it. I *also*
hesitate about the rendering-aspect, like others do, I just haven't seen
something simpler - until now.

The last few weeks I started thinking about an alternative or rather,
complimentary approach. Mike - yes, this post is long too - but it is a
full new proposal so bear with me.

Let me interweave with Jason's post:

> Ok, I've stated my reasons why I don't like Göran's approach, now time
> to give a suggestion myself:
>
> What if namespaces were just objects that behave like dictionaries
> with the key being a string and value being a meta-class?
>
> They could be used as in:
>
> Seaside at: #Component

This actually already works exactly like that in my ::-proposal and yes,
of course we should have a class Namespace and of course instances of
those should behave like Dictionaries as above. This is not new, I presume
most other proposals do the same.

> or perhaps with a special shorthand:
>
> Seaside Component   "maybe Component needs a #?"

This also works in my ::-proposal - or rather, in my dev image! Just added
a DNU handler that does it. Note though that in both these examples the
Namespace itself is still bound at compile time. And also, DNUs are costly
I think, but perhaps neglectible.

This is the "Dan-syntax" btw, and even though I like the "it is just
messages" aspect of it I really do question the readability. But it still
is not a big deal, so it doesn't matter. But newbies will doubtless wonder
what the heck is going on, I promise!

Note though that the interesting bit is *still* compile time bound in both
these examples - the ref to the namespace itself; Seaside - see below.

Well, I would actually consider going in the OPPOSITE direction inspired
by Dan etc. Why not just go *fully late bound*? It would solve lots of
problems - like dynamic remapping of spaces without any recompilation
needed.

But offering dynamic lookup and still using compile time binding TOO -
that defeats the power of it. If we ONLY use late binding then we have new
interesting opportunities. But what about the old code then? Well:

What if we could just turn a normal good old global ref into something
late bound? If we could, then old code would also turn *fully late bound*
when recompiled.

One way to do this is to compile a global ref as a message send so that:

   Array

...compiles *as if* it said:

   Smalltalk resolve: #Array

...or something similar. Then we can actually dynamically figure out if it
should evaluate to "good old Array" or "Jim's new cooler Array" depending
on for example the PI of the sender or any current remappings or whatever!

So in summary you would type:

   Array

...if you want to dynamically resolve a "global", using whatever funkiness
you want. And then you can also use more specific lookup of course:

   Seaside Component

...or of course:

   Seaside at: #Component

So this would first dynamically resolve "Seaside" and then, we ask that
object (hopefully a Namespace instance) to give us whatever it thinks is
#Component.

The reason I started thinking in this direction - getting closer to Dan's
original ideas I presume - was some older postings from Chris which he
pointed to. He more or less wrote that most problems could be solved if we
could dynamically remap packages - or in other words, dynamically put
packages in namespaces of our liking. After hacking on this in my older
proposal I came quickly to the idea that compile time binding sucks. :)

What would above need then? Well, three parts:

1. Hack Compiler and friends. Not that big a deal actually. No new syntax.

2. Implement SystemDictionary>>resolve: in some simple way. The obvious
way is to just do "self at: key", but we could perhaps add a "remap"
mechanism from the start so that you can put "redirections" in place.

3. Implement class Namespace etc, well, my proposal has the approach of
"simulating" Namespaces within SystemDictionary, if someone else has
something better that works - fine. We can even have different such
beasts.

Rendering etc goes out the window, short forms could be used if #resolve:
was made smarter (like look for "Array" in all namespaces and pick first
found), imports can be added on top, if people want to, but is not needed
etc etc.

This sounds even simpler and more backwards compatible than my first
proposal. ;)

regards, Göran

On 10/4/07, Göran Krampe <[hidden email]> wrote:
> One way to do this is to compile a global ref as a message send so that:
>
>    Array
>
> ...compiles *as if* it said:
>
>    Smalltalk resolve: #Array
>
> ...or something similar.

Some versions of Smalltalk do something like this.  The bytecodes that
get generated use the Association for the global as the literal, and
send the #value message to read the global and the #value: message to
write it.  However, I just looked at a Squeak method and it looks like
Squeak does something different, that it has special bytecodes for
reading a global.  It would be easy to change, though.

>
> What would above need then? Well, three parts:
>
> 1. Hack Compiler and friends. Not that big a deal actually. No new syntax.

> 2. Implement SystemDictionary>>resolve: in some simple way. The obvious
> way is to just do "self at: key", but we could perhaps add a "remap"
> mechanism from the start so that you can put "redirections" in place.
>
> 3. Implement class Namespace etc, well, my proposal has the approach of
> "simulating" Namespaces within SystemDictionary, if someone else has
> something better that works - fine. We can even have different such
> beasts.

> This sounds even simpler and more backwards compatible than my first
> proposal. ;)

I agree that changing the compiler but NOT changing the VM or changing
syntax is more likely to be backwards compatible, and so easier to get
people to adapt it.

-Ralph Johnson

> Some versions of Smalltalk do something like this.  The bytecodes that
> get generated use the Association for the global as the literal, and
> send the #value message to read the global and the #value: message to
> write it.  However, I just looked at a Squeak method and it looks like
> Squeak does something different, that it has special bytecodes for
> reading a global.  It would be easy to change, though.

I was wrong, Squeak does store the Association in the literal frame of
the method.  What it doesn't do is send the #value message.  Squeak
uses a class called ReadOnlyVariableBinding, which is essentially an
Association.  There is a special bytecode called pushlit: that seems
to push the value of the ROVB, but you could probably use
#pushConstant: and then send #value to it and get the same result.

If that works, all you have to do is to change the #value method to
change how class names are looked up.

-Ralph

Göran Krampe wrote:
This is trivial to do but unfortunately it doesn't solve *any* real
problems. But it creates new ones: For example, access to classes would
suddenly be slowed down by 100x or more and you'd have to be very
careful where and how you use classes (no use in tight loops for
example) then creating new kinds of references (in iVars etc) that the
compilers don't know about and can't fix when recompiling stuff.

This has always been my problem with these late bound proposals: They
sound nice, but they do not solve any actual problem that their
proponents can point to, and everyone will be paying the price for a
horribly slow lookup of classes. Bad tradeoffs.

Cheers,
   - Andreas

Forgot to point out how to trivially implement that proposal:

* Change ROVB>>isSpecialReadBinding to return true (this will send #value)

* Implement ROVB>>value as:
ROVB>>value
   ^value ifNil:[Smalltalk at: key ifAbsent:[nil]]

* Recompile the entire system (to use ROVB>>value)

* Go through the class names in Smalltalk and nil out their values:

   Smalltalk associationsDo:[:bndg| bndg value isBehavior ifTrue:[
     [bndg value: nil]
         on: AttemptToWriteReadOnlyGlobal
         do:[:ex| ex resume: true].
   ]].

Cheers,
   - Andreas

Andreas Raab wrote:

On 04/10/2007, Andreas Raab <[hidden email]> wrote: to inform compiled class that it must update its external references, if any.
I don't see the really big problems with late bound name resolution.
It will shift some of the programming techniques, and some of them in
really good way, like code/behavior locality, solving naming conflicts
e.t.c.
And default name resolution was proposed only for backward compatibility.


> This has always been my problem with these late bound proposals: They
> sound nice, but they do not solve any actual problem that their
> proponents can point to, and everyone will be paying the price for a
> horribly slow lookup of classes. Bad tradeoffs.
>
> Cheers,
>    - Andreas
>
>


--
Best regards,
Igor Stasenko AKA sig.

Funny Andreas, we could have said exactly the same about messages...
Sure a lot of curly brace defenders have already.

 From a pragmatical POV, you're probably right.
But from a historical, not sure...
Be a future VM able to transparently inline those lookups, that your
arguments might vanish.

Andreas Raab a écrit :

Aha.  I thought yours had a big security piece, and you're already
decided about imports etc.  I was thinking we could give Göran's
strategy (or something close to it) a try.

Anyway, I don't have anything against your solution.  I'm hopeful it
works out, but I'm a bit weary of the security focus.  Security is on
one side of a sliding scale and ease of use on the other.  I'm more
worried about ease of use for my dev environment.

But if it shows to be transparent to me as a developer then I think
your solution is probably the best on the table.

On 10/4/07, Michael van der Gulik <[hidden email]> wrote:

On 10/4/07, Jason Johnson <[hidden email]> wrote:
> Aha.  I thought yours had a big security piece, and you're already
> decided about imports etc.  I was thinking we could give Göran's
> strategy (or something close to it) a try.

In case this wasn't clear, I mean Göran's strategy in regards to
importing (i.e. none or at least very simple).

On 10/4/07, Göran Krampe <[hidden email]> wrote:
>
> Note though that the interesting bit is *still* compile time bound in both
> these examples - the ref to the namespace itself; Seaside - see below.

It doesn't have to be.  The syntax I expressed could be actually
resolved compile time or late bound, depending on how it was
implemented.

> Well, I would actually consider going in the OPPOSITE direction inspired
> by Dan etc. Why not just go *fully late bound*? It would solve lots of
> problems - like dynamic remapping of spaces without any recompilation
> needed.

Because of the speed concerns, and it may not (probably isn't) always
be needed.  My idea here was just implement it purely late bound with
no option for compile time binding, and just use macros (which I want
anyway) to implement early binding.

> But offering dynamic lookup and still using compile time binding TOO -
> that defeats the power of it. If we ONLY use late binding then we have new
> interesting opportunities.

You lost me here.  How does having options lose power?

> This sounds even simpler and more backwards compatible than my first
> proposal. ;)

Cool.  It sounds interesting to me as well.  I'm interested to see
what Mike (did he say we can call him that? :) has to say, as his
proposal may already do much of this.

But the aspect I really like from your original proposal and again
here is not doing the imports.  I don't know if it will scale, but I
haven't seen it tried before and it might turn out very successful.

nicolas cellier wrote:
> Funny Andreas, we could have said exactly the same about messages...
> Sure a lot of curly brace defenders have already.

Not quite. There are various real problems that you can only solve via
late-binding method names to their implementations (inheritance for
example makes little sense if you need to know whether you're going to
call a function with an Orange or an Apple instead of just generally
using a Fruit). Whether using late-binding *exclusively* solves any real
problem I'm personally not so sure. There have been numerous times where
I wished to have the ability to early-bind methods and rather recompile
everything upon a change instead of being forced to use late-binding (we
pay a major price for this; both in terms of runtime and development
costs because in the case where we need it we have to punt and use C
instead).

But late-binding class names? I'm sorry, but I really fail to see which
problem this is supposed to address (and I have the distinct feeling
neither do you because otherwise you probably would have pointed out at
least one good reason why it is advantageous ;-) Besides, I would like
to point out that we often use late-bound classes already (browse those
implementors of #controllerClass in MVC) and that classes are
automatically kept up-to-date (which would be a good reason for
late-binding them otherwise). And as if that weren't enough you can just
assign them if you'd like to use some other class instead. So, once
again, what is the advantage of late-binding class names? And how does
it outweigh the price you'd have to pay?

> From a pragmatical POV, you're probably right.
> But from a historical, not sure...
> Be a future VM able to transparently inline those lookups, that your
> arguments might vanish.

In which case I'll happily late-bind them. Because if there is no cost
associated with it then there's really no reason not to (outside
security). I have no general problem with late-binding class names
(security concerns excluded) but given the price, I'd like to know why
exactly I should be paying it.

Cheers,
   - Andreas

Jason Johnson wrote: But I have always been impressed with Self's philosophy: design it the
right way, and let the implementation find the fast way to do it, behind
the scenes.  So, I think late-binding is the right way to specify this
stuff.  The implementors need to solve the speed problem.   (easy for me
to say ;-) )

David

I saw, there are already present an implementation of PICs for squeak,
made available for use with bytecoded methods with slightly changing
the method message lookup and activating sequence.
Why its still not used/adopted by squeak (i suppose i missed
discussion about this)?

On 05/10/2007, David P Harris <[hidden email]> wrote:

--
Best regards,
Igor Stasenko AKA sig.

>> Well, I would actually consider going in the OPPOSITE direction inspired
>> by Dan etc. Why not just go *fully late bound*? It would solve lots of
>> problems - like dynamic remapping of spaces without any recompilation
>> needed.
>
> Because of the speed concerns, and it may not (probably isn't) always
> be needed.  My idea here was just implement it purely late bound with
> no option for compile time binding, and just use macros (which I want
> anyway) to implement early binding.

I think that late-binding Undefined references would be much preferrable
to early-binding them inside a separate dictionary.  However, for things
that "exist" I don't think early-binding loses flexibility compared to
late binding.

Paolo

I can only agree with your wisdom.
Neither would I pay the 100x speed down for nothing.

However, the fact that we rarely use class late binding is not a proof
it wouldn't be usefull. Maybe we don't use it because we are not used to it.

Beside the good Undeclared example of Paolo, it could for example enable
executing some code safely by changing all dangerous class references
with secured restricted versions. Certainly easier if late bound.

I think most applications would occur when playing with packages
loading/unloading/ replacing with an experimental version, or maybe
implementing an auto-loading feature more easily.

Note that VW has introduced a special syntax for late binding
#{ThisIsALateBindingExample}.
This must be to accomodate increased complexity brought by namespaces I
guess and/or the loading of Parcels/Packages.
That's precisely the point. Scaling up the image, imitating what the OS
does with applications and libraries. There could lie the major use of
late binding.

Whether we should or not follow this path is another debate.
You can deplore that "Small"talk is getting too big, and is loosing its
soul: code is becoming less factored. With more than 2000 classes, the
probability for rewriting some code already in the image is for sure
greater than with 200.
See the enlightening example of Net::etc...::Socket duplication from the
namespace thread...

That's a fact. Image is growing.
Pushing more packages out is likely to create more problem than solved,
more duplication, less factoring, thus the namespace debate and its
class-late-binding avatar...


Andreas Raab a écrit :

Gulik.

--
http://people.squeakfoundation.org/person/mikevdg
http://gulik.pbwiki.com/