Musings about modularity and programming in the large

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
41 messages Options
123
Reply | Threaded
Open this post in threaded view
|

Re: Musings about modularity and programming in the large

stephane ducasse
>
>       1. Orthogonal extension: Extensions may require both new data  
> types and new operations.
>       2. Type safety: Extensions cannot create run-time type errors.
>       3. Modularity: The base system can be extended without  
> modifying or recompiling its code.
>       4. Scalability: Extensions should be scalable. The amount of  
> code needed should be proportional to the functionality added.
>       5. Non-destructive extension: The base system should still be  
> available for use within the extended system.
>       6. Composability of extensions.
>
>    The first three of these requirements correspond to Wadler’s  
> expression problem. Scalability (4) is often but not necessarily  
> satisfied by supporting separate compilation; it is important for  
> extending large software. Non-destructive extension (5) enables  
> existing clients of the base system and also the extended system  
> itself to interoperate with code and data of the base system, an  
> important requirement for backward compatibility. Nested inheritance  
> addresses the first five requirements, but it does not support  
> extension composition. Nested intersection adds this capability.
> "
>
>
> IMO point 5 is often overlooked in Smalltalk systems, we love  
> extending base code but we hardly ever think about isolating these  
> extensions so the base code dependents don't get 'confused' by those  
> extensions.


classboxes was akind of answer to this point. But I do not really like  
their per thread semantics. So now I would favor extensions = addition  
but no override
but the system would be less agile and breakable.
Reply | Threaded
Open this post in threaded view
|

Re: Musings about modularity and programming in the large

Ignacio Vivona-2
In reply to this post by stephane ducasse
why is not having metaclasses a nice feature?

On Jan 24, 2008 5:02 AM, stephane ducasse <[hidden email]> wrote:
yes newspeak has some nice features: mixin, no metaclass (I imagine),
"block structure" not sure how to describe that for objet
initialization...

On Jan 22, 2008, at 10:36 PM, Michael Haupt wrote:

> Hi Jason,
>
> On Jan 22, 2008 6:36 PM, Jason Johnson <[hidden email]>
> wrote:
>> So no one knows of a system like this?  There's nothing new in our
>> field since decades so I'm sure there are papers or an implementation
>> out there, I just haven't found it yet.
>
> reading your posting, I had the impression there might be some
> relations to Gilad Bracha's work on Newspeak...
>
> http://gbracha.blogspot.com/
>
> He's got several entries in his blog that describe how Newspeak deals
> with modularity, classes, dependencies, ... maybe you could give that
> a try?
>
> Best,
>
> Michael
>
>





Reply | Threaded
Open this post in threaded view
|

Re: Musings about modularity and programming in the large

Sophie424
"Ignacio Vivona" <[hidden email]> wrote in message
news:[hidden email]...
> why is not having metaclasses a nice feature?

It is about metaclasses not being "special" or hidden or singletons, but
being just as user-accessible, definable, instantiable, and extensible as
any other object (and class). It is a much simpler meta-structure than
Smalltalk-80.

e.g. See
http://www.iam.unibe.ch/~ducasse/Web/ArchivedLectures/p156-cointe.pdf.

NewSpeak seems to incorporate first-class namespaces and modules within this
really clean and simple framework.

- Sophie

================
(If interested, here is a quote and a summary of that paper):

"This paper shows how an attempt at a uniform and reflective definition
resulted in an open-ended system ... We propose to unify Smalltalk classes
and their terminal instances. This unification allows us to treat a class as
a "first class citizen", to give a circular definition to the first
metaclass, to access to the metaclass level, and finally to control the
instantiation link. Because each object is an instance of another one and
because a metaclass is a real class inheriting from another one, the
metaclss links can be created indefinitely."

Summary (all errors mine :-)
- Just 2 pre-defined classes: Object and Class
  - every object (including all classes) has instVars and behaviors
- Just 5 pre-defined instVars:
  All objects:
  - instanceOf: every object is instanceOf some class
  All classes:
  - name: name of class
  - supers: every class (except Object) has 1(+) super class
  - instanceVariables (of its instances)
    instanceMethods (method dictionary of its instances)
     - every class has these
     - these control instVars and behaviors of instances of that class
- Just 2 pre-defined behaviors:
  - #send:args: every object supports this
  - #basicNew: every class supports the #basicNew primitive
     - #basicNew on a metaclass produces another class
  - A terminal (i.e. non-class) object does not respond to #new
- Class is subclass of Object
  - so every class is an object
- Class is an instance of Class
  - so the instVars and behaviors of Class are defined by Class
  - Class is the initial primitive "metaclass"
- Object is an instance of Class

- Meta-stuff is now nothing special
  - "As a common object, a class is defined by its class
     and the values of its associated instances variables", no more
  - Any subclass (direct or indirect) of Class is a meta-class




Reply | Threaded
Open this post in threaded view
|

Re: Musings about modularity and programming in the large

Andreas.Raab
itsme213 wrote:

> "Ignacio Vivona" <[hidden email]> wrote in message
> news:[hidden email]...
>> why is not having metaclasses a nice feature?
>
> It is about metaclasses not being "special" or hidden or singletons, but
> being just as user-accessible, definable, instantiable, and extensible as
> any other object (and class). It is a much simpler meta-structure than
> Smalltalk-80.
>
> e.g. See
> http://www.iam.unibe.ch/~ducasse/Web/ArchivedLectures/p156-cointe.pdf.

I'll have to admit I never liked this model too much. One of the killer
features of the Smalltalk-80 model is that it allows you to add features
to the meta class easily (by just clicking on the meta-tab in the
browser) be that class instance variables, instances creation, utility,
example, or test methods. That is something that although I'm in favour
of simplicity I wouldn't want to live without and if you look at usage
patterns, like:

n := c := 0.
Smalltalk allClassesDo:[:cls|
        n := n + 1.
        cls selectors isEmpty ifFalse:[c := c + 1].
].
c asFloat / n asFloat.

you will find that 90+ per cent of the classes in a random image
actually make use of their private meta class. But once you agree that
these are desirable features, there is only so many ways in which you
can do them.

I think the main question here is: How often do people extend their
meta-classes by a few simple utility methods vs. how often do they do
more elaborate changes that can't be represented well in the ST-80
Metaclass hierarchy? Given the data points it appears to me that Dan et
al. got it just about exactly right.

Cheers,
   - Andreas

Reply | Threaded
Open this post in threaded view
|

Re: Musings about modularity and programming in the large

Andreas.Raab
Andreas Raab wrote:
> n := c := 0.
> Smalltalk allClassesDo:[:cls|
>     n := n + 1.
>     cls selectors isEmpty ifFalse:[c := c + 1].
> ].
> c asFloat / n asFloat.

Oops, how embarrassing. It should be "cls class selectors isEmpty" and
this reduces the number of classes that utilize their metaclass to
roughly 50% (which is still huge).

Cheers,
   - Andreas


Reply | Threaded
Open this post in threaded view
|

Re: Musings about modularity and programming in the large

Sophie424
In reply to this post by Andreas.Raab
"Andreas Raab" <[hidden email]> wrote

>> It is about metaclasses not being "special" or hidden or singletons, but
>> being just as user-accessible, definable, instantiable, and extensible as
>> any other object (and class). It is a much simpler meta-structure than
>> Smalltalk-80.
>
> I'll have to admit I never liked this model too much. One of the killer
> features of the Smalltalk-80 model is that it allows you to add features
> to the meta class easily (by just clicking on the meta-tab in the browser)
> be that class instance variables, instances creation, utility, example, or
> test methods.

I agree that is extremely convenient. But could it be thought of as good
tool / browser design? Even on Cointe's model, could a tool that knows it is
editing a class make it just as painless to
  1. select a metaclass
  2. create a metaclass on the fly
  3. do 1 & 2 with on-the-fly customizations to ((meta)class)instVars,
methods...

(Ruby's instance-level customizations do things similar to 1 & 2; our
instances would be classes).

I can imagine (newbie alert :-), a similar "class" tab with similar
convenient behaviors. Plus the added benefit, specially for those interested
in meta-modeling and DSLs, of all full access to any levels of "metaclass"
definitions.

Sophie




Reply | Threaded
Open this post in threaded view
|

Re: Musings about modularity and programming in the large

Andreas.Raab
itsme213 wrote:

>> I'll have to admit I never liked this model too much. One of the killer
>> features of the Smalltalk-80 model is that it allows you to add features
>> to the meta class easily (by just clicking on the meta-tab in the browser)
>> be that class instance variables, instances creation, utility, example, or
>> test methods.
>
> I agree that is extremely convenient. But could it be thought of as good
> tool / browser design? Even on Cointe's model, could a tool that knows it is
> editing a class make it just as painless to
>   1. select a metaclass
>   2. create a metaclass on the fly
>   3. do 1 & 2 with on-the-fly customizations to ((meta)class)instVars,
> methods...

Not unless you really recreate the full ST-80 model. There is a ton of
subtleties in the design which is hard to recreate properly. For
example, a while back I wrote a self-contained metaclass kernel (if you
search the archives you may find it) but as soon as you introduce new
Metaclasses in the structure you are breaking the inheritance of the
metaclass hierarchy. And while this may be acceptable in a few
well-defined situations (like for bootstrapping a complete
self-contained kernel) it's certainly nothing that one would like to
deal with in daily work.

Cheers,
   - Andreas

> (Ruby's instance-level customizations do things similar to 1 & 2; our
> instances would be classes).
>
> I can imagine (newbie alert :-), a similar "class" tab with similar
> convenient behaviors. Plus the added benefit, specially for those interested
> in meta-modeling and DSLs, of all full access to any levels of "metaclass"
> definitions.
>
> Sophie
>
>
>
>
>


Reply | Threaded
Open this post in threaded view
|

Re: Musings about modularity and programming in the large

Michael van der Gulik-2
In reply to this post by Jason Johnson-5


On Jan 24, 2008 7:02 PM, Jason Johnson <[hidden email]> wrote:
On Jan 23, 2008 11:50 PM, Michael van der Gulik <[hidden email]> wrote:
> http://gulik.pbwiki.com/Namespaces
>
> There is code available on the 3.10 Package Universes that I'm currently
> using to varying degrees of success.
>
> Gulik.

Well, this isn't what I had in mind [1], but it looks quite good I
must say.  Are you planning to keep this namespace system as a
separate entity from "User" and the other concepts you mentioned?  


Yes. Namespaces are just another component of my Unnamed Grand Project[1][2].

[1] http://gulik.pbwiki.com/Unnamed+Grand+Project
[2] Naming suggestions welcome.

 
The
namespace part seems like potentially a good candidate for adoption to
me.  Do you have any pointers to projects using it, to get a feel for
how well it's working in practice?


It won't be a candidate for adoption until the implementation is stable and widely used.

As of yet, I'm only beginning to use my own tools to make actual namespaced projects. If you load the namespace tools from the 3.10 package universe, it automatically generates some demo namespaces (Kernel, Collection), but I wouldn't recommend using these in your own projects except for playing around. I'm currently porting Kernel and Collection to namespaces now.

 
[1] It's actually not so far away though.  What I was envisioning was
a system that behaves mostly like now, but "module classes" or
"component classes" (classes that are made up of *classes* and methods
instead of just methods) would be visible in the main dictionary and
none of the classes that make one up would be visible.  The only way
one could be referenced outside is if the "module class" returned on
from a method.  There would probably be on the code browser when
looking at such a class that lets one "zoom in" to see inside it and
make changes.


This sounds like a facade pattern to me.

I'm considering making namespaces able to be "private". You could then make a facade object which was the only class that has access to that namespace.

Gulik.

--
http://people.squeakfoundation.org/person/mikevdg
http://gulik.pbwiki.com/

Reply | Threaded
Open this post in threaded view
|

Re: Musings about modularity and programming in the large

Sophie424
In reply to this post by Andreas.Raab
(All this may be painfully obvious to others, and this is somewhat off-topic
from the original post, apologies ...)

>"Andreas Raab" <[hidden email]> wrote Not unless you really recreate
>the full ST-80 model. There is a ton of subtleties in the design which is
>hard to recreate properly.

I'm sure I overlook tons of stuff :)

> ... but as soon as you introduce new Metaclasses in the structure you are
> breaking the inheritance of the metaclass hierarchy.

If you will bear with me ... perhaps you mean this in the sense of the
"Uniform and safe metaclass composition" paper by Stephane et al. Ruby does
object-level customization in a way which retains the inheritance structure,
roughly as follows:

x = String.new
class << x
  def dance_and_sing
    ...
  end
end

x.dance_and_sing

- if object x is instanceOf (a regular, non-singleton) class C
- customizing object x (e.g. dance_and_sing above) will
  - insert an anonymous singleton class xC' _below_ C
     (if needed; not if x already had a singleton class)
  - put x customomizations into xC'
  - x is still (indirectly) an instance of the original C

Would something like that be relevant to not breaking the inheritance of the
metaclass hierarchy?

- Sophie




Reply | Threaded
Open this post in threaded view
|

Re: Musings about modularity and programming in the large

Igor Stasenko
On 24/01/2008, itsme213 <[hidden email]> wrote:

> (All this may be painfully obvious to others, and this is somewhat off-topic
> from the original post, apologies ...)
>
> >"Andreas Raab" <[hidden email]> wrote Not unless you really recreate
> >the full ST-80 model. There is a ton of subtleties in the design which is
> >hard to recreate properly.
>
> I'm sure I overlook tons of stuff :)
>
> > ... but as soon as you introduce new Metaclasses in the structure you are
> > breaking the inheritance of the metaclass hierarchy.
>
> If you will bear with me ... perhaps you mean this in the sense of the
> "Uniform and safe metaclass composition" paper by Stephane et al. Ruby does
> object-level customization in a way which retains the inheritance structure,
> roughly as follows:
>
> x = String.new
> class << x
>   def dance_and_sing
>     ...
>   end
> end
>
> x.dance_and_sing
>
> - if object x is instanceOf (a regular, non-singleton) class C
> - customizing object x (e.g. dance_and_sing above) will
>   - insert an anonymous singleton class xC' _below_ C
>      (if needed; not if x already had a singleton class)
>   - put x customomizations into xC'
>   - x is still (indirectly) an instance of the original C
>
> Would something like that be relevant to not breaking the inheritance of the
> metaclass hierarchy?
>

I heard about this a while ago. IIRC perl having per-object behavior
customization.
But this can be easily made , when object model is prototype based
(perl/javascript).
And in ST we having a class-based model.
Btw, can you show us a 'killer' use case of such language feature,
which shows great benefit comparing to class-based model? I'm very
intrigued. :)


--
Best regards,
Igor Stasenko AKA sig.

Reply | Threaded
Open this post in threaded view
|

Re: Musings about modularity and programming in the large

Jason Johnson-5
In reply to this post by Michael van der Gulik-2
On Jan 24, 2008 9:53 PM, Michael van der Gulik <[hidden email]> wrote:
>
> Yes. Namespaces are just another component of my Unnamed Grand
> Project[1][2].
>
> [1] http://gulik.pbwiki.com/Unnamed+Grand+Project
> [2] Naming suggestions welcome.

Wonderful.  And I think naming is the hardest part of programming.

> It won't be a candidate for adoption until the implementation is stable and
> widely used.

Sorry, I meant "candidate" as in "something we can watch and when it
is ready consider it for adoption".

Reply | Threaded
Open this post in threaded view
|

Re: Musings about modularity and programming in the large

Jason Johnson-5
In reply to this post by Igor Stasenko
On Jan 25, 2008 1:08 AM, Igor Stasenko <[hidden email]> wrote:
>
> I heard about this a while ago. IIRC perl having per-object behavior
> customization.
> But this can be easily made , when object model is prototype based
> (perl/javascript).

Perl has the almost exactly the same OO system Lua has, i.e. nearly
nothing.  In both cases, OO systems are added as libraries (you're
probably thinking of one called Moose).  But perl's "object model" is
not prototype based, just some of the modules on CPAN duck tape such a
model on.

The prototype method of OO is interesting.  It's a shame the best
incarnation of it, Self, seems to be defunct at the moment.  I hope it
comes back some day.

Reply | Threaded
Open this post in threaded view
|

Re: Musings about modularity and programming in the large

Jason Johnson-5
In reply to this post by stephane ducasse
On Jan 24, 2008 8:57 AM, stephane ducasse <[hidden email]> wrote:
> Wy
>
> Module named: #Zork
>         contains: { A, B }

Ok, you're right.  You don't *have* to.  But every proposal I have
seen does in some way (usually importing or specifying which version
of a class you actually meant).

> > As far as confusion, that is a valid concern, but in modern times all
> > development is done in an IDE.  That is, the tools expand our
> > capabilities and I would rely on the tools for this situation as well.
> > The tools would have to operate such that there was no confusion.
>
> I do not believe that. I'm teaching too much oop to eat that cake.

I think you misunderstand me here.  I don't mean the "when I press
this button, please dump these 30 lines of code into my source"
nonsense that one does in a Java IDE.  I was thinking more about
things like Shout that make it faster to digest what you're seeing in
source code.

Reply | Threaded
Open this post in threaded view
|

Re: Musings about modularity and programming in the large

Igor Stasenko
In reply to this post by Jason Johnson-5
On 25/01/2008, Jason Johnson <[hidden email]> wrote:

> On Jan 25, 2008 1:08 AM, Igor Stasenko <[hidden email]> wrote:
> >
> > I heard about this a while ago. IIRC perl having per-object behavior
> > customization.
> > But this can be easily made , when object model is prototype based
> > (perl/javascript).
>
> Perl has the almost exactly the same OO system Lua has, i.e. nearly
> nothing.  In both cases, OO systems are added as libraries (you're
> probably thinking of one called Moose).  But perl's "object model" is
> not prototype based, just some of the modules on CPAN duck tape such a
> model on.
>
> The prototype method of OO is interesting.  It's a shame the best
> incarnation of it, Self, seems to be defunct at the moment.  I hope it
> comes back some day.
>
>
Ok, but the question, what gives per-object on-the-fly behavior
customization comparing to strict class model still is unanswered.
I agree, its more flexible than classes, but i don't see real use
cases, where such feature, when introduced, gives more flexibility in
project development.

As analogue, in ST, when i want some code to behave differently,
depending on situation, i always can use blocks , to represent my
custom code.
But i don't see, how extending/replacing behavior without using
classes (on per-object basis) can be really better than subclassing
from dev's point of view and from POV of project integrity and it's
readability.

--
Best regards,
Igor Stasenko AKA sig.

Reply | Threaded
Open this post in threaded view
|

Re: Musings about modularity and programming in the large

Sophie424
In reply to this post by Igor Stasenko
"Igor Stasenko" <[hidden email]> wrote in message

>> Would something like that be relevant to not breaking the inheritance of
>> the
>> metaclass hierarchy?
>>
>
> I heard about this a while ago. IIRC perl having per-object behavior
> customization.
> But this can be easily made , when object model is prototype based
> (perl/javascript).
> And in ST we having a class-based model.
> Btw, can you show us a 'killer' use case of such language feature,
> which shows great benefit comparing to class-based model? I'm very
> intrigued. :)

Ruby is a class-based, not a prototype-based language. It allows per-object
customization by silently inserting a per-object class only where needed,
but does not go the next step with the required "self" magic of a
prototype-based language. Smalltalk can do this easily (perhaps not as
conveniently as Ruby) e.g. Avi has a package on SqueakSource called
Prototype, I think.

More importantly, I was not pushing prototype-based languages at all. I was
simply asking if the Ruby way of inserting an anonymous class could be used
safely at the meta-level:
  - Every class C is an instance of its own class CC, which is Class by
default
  - Every "class-side" customization of C (class inst-vars, class methods,
etc.) will either
       - customize the explicit CC (named, shareable, etc.) meta-class, or
       - insert an anonymous subclass of CC, and then customize that

If something like this worked, it could retain most of the convenience of
the simple instance/class browser tabs for many users, and still allow
full-blown meta-class manipulation where needed.

- Sophie




Reply | Threaded
Open this post in threaded view
|

Re: Musings about modularity and programming in the large

Bert Freudenberg

On Jan 25, 2008, at 15:48 , itsme213 wrote:

> "Igor Stasenko" <[hidden email]> wrote in message
>>> Would something like that be relevant to not breaking the  
>>> inheritance of
>>> the
>>> metaclass hierarchy?
>>>
>>
>> I heard about this a while ago. IIRC perl having per-object behavior
>> customization.
>> But this can be easily made , when object model is prototype based
>> (perl/javascript).
>> And in ST we having a class-based model.
>> Btw, can you show us a 'killer' use case of such language feature,
>> which shows great benefit comparing to class-based model? I'm very
>> intrigued. :)
>
> Ruby is a class-based, not a prototype-based language. It allows  
> per-object
> customization by silently inserting a per-object class only where  
> needed,
> but does not go the next step with the required "self" magic of a
> prototype-based language. Smalltalk can do this easily (perhaps not as
> conveniently as Ruby) e.g. Avi has a package on SqueakSource called
> Prototype, I think.


FWIW this is exactly what Etoys does (which implements a fat object  
model on top of Squeak).

- Bert -



Reply | Threaded
Open this post in threaded view
|

Re: Musings about modularity and programming in the large

Sophie424
In reply to this post by stephane ducasse
"stephane ducasse" <[hidden email]> wrote
> a class = factory of objects, a modules = scoped group  of classes is a
> nice distinction.

Could ModuleInstance be very useful? What do you think of how NewSpeak
defines
 module = class (in top-level namespace)
   scoped group of classes normally accessed via methods on module instance
 module instance = module (class) instantiated e.g. with library bindings




Reply | Threaded
Open this post in threaded view
|

Re: Musings about modularity and programming in the large

Jason Johnson-5
In reply to this post by Igor Stasenko
On Jan 25, 2008 9:01 AM, Igor Stasenko <[hidden email]> wrote:
>
> Ok, but the question, what gives per-object on-the-fly behavior
> customization comparing to strict class model still is unanswered.

I don't have that answer.

> I agree, its more flexible than classes

Well, I'm not convinced that it's more flexible then classes.  I said
it was interesting, and I would like to explore it to find out what
the pluses and minuses are.  But since I don't know of a running self
implementation I can only imagine.

Reply | Threaded
Open this post in threaded view
|

Re: Musings about modularity and programming in the large

Jason Johnson-5
On Jan 25, 2008 5:40 PM, Jason Johnson <[hidden email]> wrote:
> But since I don't know of a running self
> implementation I can only imagine.

Correction:  I don't know of a running Self implementation for a
platform I have (I don't yet have a Mac).

Reply | Threaded
Open this post in threaded view
|

Re: Musings about modularity and programming in the large

Klaus D. Witzel
In reply to this post by Jason Johnson-5
On Fri, 25 Jan 2008 17:40:11 +0100, Jason Johnson wrote:

> On Jan 25, 2008 9:01 AM, Igor Stasenko wrote:
>>
>> Ok, but the question, what gives per-object on-the-fly behavior
>> customization comparing to strict class model still is unanswered.
>
> I don't have that answer.
>
>> I agree, its more flexible than classes
>
> Well, I'm not convinced that it's more flexible then classes.  I said
> it was interesting, and I would like to explore it to find out what
> the pluses and minuses are.  But since I don't know of a running self
> implementation I can only imagine.

You might want to use one of these

- http://gliebe.de/self/download.html

>



123