Spoon progress 23 May 2006: finding dependencies via passive imprinting

On May 25, 2006, at 1:08 AM, Craig Latta wrote:

Heh, yeah. Symbols are explicitly designed so that name and identity  
are *not* distinct.

One more question, though. How does Spoon maintain Selector identity  
across object memories? When a method is transferred from one object  
memory to another, how are the selectors in its literal frame  
interned when it arrives? I don't imagine the selectors have  
UUIDs.... perhaps the selector tables do?

Thanks,

Colin

Hi Ralph--

 > I'm just getting into Spoon (hmm, I'm just learning to use Spoon, I'm
 > getting stirred up with Spoon, ... there ought to be a good line
 > here)...

        Yeah, I like "getting a handle on Spoon". :)

        Spoon currently still has the old system dictionary, but I'd like to
remove it. I think we should just rewrite users of "Smalltalk at:"
(probably in some automated fashion, both in situ and during method
transfer). Instead of a system dictionary, I'd rather have some class
take responsibility for each former non-class "global" variable, and use
messages for access. Or just use shared pools for such variables. (Heh,
shared pools is another topic that gets some people riled up. :)

        The classes we can put in some cache collection that is known to the
compiler, but it need not be keyed. Then each class has complete
responsibility for its name, and we needn't worry about other objects
having out-of-sync notions of any class names.

        I don't think we need class or module namespaces in Spoon.


-C

--
Craig Latta
improvisational musical informaticist
www.netjam.org
Smalltalkers do: [:it | All with: Class, (And love: it)]

Hi Colin--

 > How does Spoon maintain Selector identity across object memories? When
 > a method is transferred from one object memory to another, how are the
 > selectors in its literal frame interned when it arrives? I don't
 > imagine the selectors have UUIDs.... perhaps the selector tables do?

        There's no need to maintain Selector identity across object memories.
The only important thing is that a Selector used in the literal frame of
a sending method be the same one used as the method dictionary key for
the method to which the sending method refers. The behavior for
transferring methods ensures this.


        thanks,

-C

--
Craig Latta
improvisational musical informaticist
www.netjam.org
Smalltalkers do: [:it | All with: Class, (And love: it)]

On 5/26/06, Craig Latta <[hidden email]> wrote:
>         Spoon currently still has the old system dictionary, but I'd like to
> remove it. I think we should just rewrite users of "Smalltalk at:"
> (probably in some automated fashion, both in situ and during method
> transfer).

And how do you propose to do this?  The reason I posted my message was
that I couldn't see any way that Spoon could implement "Smalltalk at:"
and I thought that there should be a way.  From what you wrote below,
perhaps you want to convert "Smalltalk at: className" into "Object
allSubclasses detect: [:eachClass | eachClass name = className]".
This will be nondeterministic when you have multiple classes with the
same name.

>Instead of a system dictionary, I'd rather have some class
> take responsibility for each former non-class "global" variable, and use
> messages for access.

This is easy for non-class globals.  Make them Singletons.

The problem remains, though, what about classes?

>         The classes we can put in some cache collection that is known to the
> compiler, but it need not be keyed. Then each class has complete
> responsibility for its name, and we needn't worry about other objects
> having out-of-sync notions of any class names.

So, you want to find a class by sequential search, asking each class
for its name?

Do you want to search the subclasses of Object, or ProtoObject?  Or
would you rather have a separate object to be this cache?  If you have
a separate object, how do you add objects to it when you load a
module?

-Ralph Johnson

On May 26, 2006, at 4:31 AM, Craig Latta wrote:

Let me give an example.

Lets say I've got some GIS code, and I overload #at:put:. So 1-
at:put: refers to the usual meaning for Arrays and Dictionaries,  
while 2-at:put: is my new meaning for plotting locations on maps. My  
GIS module uses both. Let's say that a class implementing 2-at:put:  
gets transferred, and then later, a method that sends 2-at:put:. So  
we've got 2-at:put: going across the wire in 3 places: as a key in a  
method dictionary, in the literal frame of a compiled method, and  
during active imprinting, a request for a missing method implementation.

Somehow, Spoon has to disambiguate between 1-at:put: and 2-at:put: as  
they move across the wire, so that selectors in literal frames  
correspond to the selectors in method dictionaries the same way they  
did in the development image. This is what I mean by "maintain  
Selector identity." How does Spoon accomplish it?

Colin

Craig

I have the impression that your Selector is a bit like  
selectorNamespace of SmallScript?
David simmon introduced a different Symbol equality.
Stef


On 25 mai 06, at 07:08, Craig Latta wrote:

Hi Ralph--

 > > Spoon currently still has the old system dictionary, but I'd like to
 > > remove it. I think we should just rewrite users of "Smalltalk at:"
 > > (probably in some automated fashion, both in situ and during method
 > > transfer).
 >
 > And how do you propose to do this?

        It would proceed similarly to what happens when an author attempts to
compile some source that uses an overloaded class name. When we
encounter an occurrence of "Smalltalk at:", we can find all the classes
in the system which have the given name. We don't have to use
 >>allSubclasses since we'll have a cache of all classes in the system
(recall my proposed replacement for the current system dictionary).

        If there is more than one class with the given name, we can ask the
human to disambiguate, showing further information about the matching
classes (author, module, etc.). If one of the choices is in a "kernel"
module (as opposed to a third-party module), that could be the default.

        I'll note here that I think "Smalltalk at:" is poor style in the first
place.

 > > The classes we can put in some cache collection that is known to the
 > > compiler, but it need not be keyed. Then each class has complete
 > > responsibility for its name, and we needn't worry about other
 > > objects having out-of-sync notions of any class names.
 >
 > So, you want to find a class by sequential search, asking each class
 > for its name?

        Yes, because class names are dynamic. I don't think the time hit is a
big deal, either. I assume you do?

 > Do you want to search the subclasses of Object, or ProtoObject? Or
 > would you rather have a separate object to be this cache?

        The latter (as I mentioned previously). And now that you mention
ProtoObject... :)  Spoon has no need for ProtoObject; in particular,
Spoon's proxy class works with VM assistance and its superclass can be
anything. I intend to remove ProtoObject.

 > If you have a separate object, how do you add objects to it when you
 > load a module?

        When you load a module, what you're actually doing is asking a new,
empty, local module to synchronize itself with a remote module that has
what you want (see Module>>synchronizeWith:). This effectively
synchronizes the two systems which host the modules, with regard to the
desired module content. During this synchronization, new classes which
are defined in the local system would be added to the class cache by the
local module.


        thanks,

-C

--
Craig Latta
improvisational musical informaticist
www.netjam.org
Smalltalkers do: [:it | All with: Class, (And love: it)]

Hi Colin--

 > Let's say I've got some GIS code, and I overload #at:put:. So
 > 1-at:put: refers to the usual meaning for Arrays and Dictionaries,
 > while 2-at:put: is my new meaning for plotting locations on maps.

        I'll just add here that if you're actually coming up with a new
conceptual meaning, from a sender's point of view, it's better (by
virtue of intelligibility) to just use a differently-named selector. I'd
anticipate overloaded selectors being useful for having multiple
implementations of the same conceptual action (e.g., for performance
reasons).

 > My GIS module uses both. Let's say that a class implementing
 > 2-at:put: gets transferred...

        "A" class? Don't you mean *the* class?

 > ...and then later, a method that sends 2-at:put:. So we've got
 > 2-at:put: going across the wire in 3 places: as a key in a method
 > dictionary...

        Note that, normally, during imprinting, when a class is defined
remotely, no additions are made to the new method dictionary at that
time. Methods are defined only later, as they are needed. An exception
to this is when the class contains overrides; then method
placeholders[1] are added.

        If an involved Selector is overloaded (not interned in the default
selector table), the providing system tells the requesting system this
as part of installing the placeholder. The receiving system can then
ensure that the Selector it uses for a method dictionary key is not
interned in the default Selector table.

 > ...in the literal frame of a compiled method, and during [passive]
 > imprinting, a request for a missing method implementation.
 >
 > Somehow, Spoon has to disambiguate between 1-at:put: and 2-at:put: as
 > they move across the wire, so that selectors in literal frames
 > correspond to the selectors in method dictionaries the same way they
 > did in the development image. This is what I mean by "maintain
 > Selector identity." How does Spoon accomplish it?

        When a providing system transfers a method with an overloaded Selector
in the literal frame, it uses a special literal marker to transfer the
Selector when transferring the method's literals. See the class comment
for the MethodLiteralTransmissionMarker class in Spoon for more about
literal markers. Basically, they're used to transfer "special" method
literals, like class pool associations.

        When a system requests a method with an overloaded Selector, it
communicates the sending method's signature as part of the request. The
providing system can then find the appropriate Selector in its own
memory (from the literal frame of its own copy of the sending method),
and thus look up the correct method to define.

        If the sending method's signature is itself overloaded, the requesting
system provides a sending chain of method signatures, ending with a
non-overloaded method signature. The providing system can then proceed
as above. This scheme won't work if the only non-overloaded method
signature in the sender graph is an unbound method, but that never
happens in an imprinting situation. There is other metadata associated
with methods (by modules) that I could use for resolution in the
providing system, like author, but I don't think I need to.

        Spoon's remote messaging protocol supports sending remote messages in
the middle of sending other messages, nested arbitrarily deeply, amongst
an arbitrary number of systems. For example, for system A to answer a
message sent by system B, it might in turn need to send remote messages
to B and C, which in turn might need to send still more messages to A.
And each parameter of each message can be a proxy on a different
arbitrary system.

        So, if necessary, there can be a quite complex back-and-forth
conversation between multiple systems in order to accomplish a
particular message-send. This functionality is needed for remote
debugging (it's why I wrote it), but I don't think it's needed for this
overloaded-selector problem. But it's available in case we come across
some pathological case I haven't thought of yet. :)

        Also note that each Selector has its own version sequence (recall that
I store a 15-bit version number in the trailer of each compiled method).
I.e., if the 1-at:put: method has version number 1, with no 2-at:put: in
existence, and I create a 2-at:put: method, the new 2-at:put: method
will have version number 1, not 2.

        Phew. :)


        thanks,

-C

[1]

http://lists.squeakfoundation.org/pipermail/squeak-dev/2006-May/103418.html

--
Craig Latta
improvisational musical informaticist
www.netjam.org
Smalltalkers do: [:it | All with: Class, (And love: it)]

On 5/27/06, Craig Latta <[hidden email]> wrote:
>
> .When we
> encounter an occurrence of "Smalltalk at:", we can find all the classes
> in the system which have the given name. We don't have to use
>  >>allSubclasses since we'll have a cache of all classes in the system
> (recall my proposed replacement for the current system dictionary).

I don't understand your replacement for the current system dictionary.
 It is just an unkeyed collection of classes.  Why bother, when you
can already ask Class for its instances, or Object for its subclasses?
 Why duplicate functionality?

I can see how to make a UI for asking someone to disambiguate a name;
that is not a problem for me.

The performance of sequencially searching for a class is also not a
problem for me.  It is not a problem in the browser, since few methods
will refer to more than a handful of classes.  If you are filing in a
lot of code, the system could make a dictionary from names to classes
and reuse it during the fileIn.

>         Yes, because class names are dynamic. I don't think the time hit is a
> big deal, either. I assume you do?

Class names are not dynamic if you want source code to work.  If you
change the name of a class then you have to find code that refers to
the class and patch it.  If you only care about compiled code, and not
source code, then class names are dynamic.

-Ralph

> From: Ralph Johnson
> I don't understand your replacement for the current system dictionary.
>  It is just an unkeyed collection of classes.  Why bother, when you
> can already ask Class for its instances, or Object for its subclasses?
>  Why duplicate functionality?

Prevention of garbage collection might be one example.

                - Peter

>
> Class names are not dynamic if you want source code to work.  If you
> change the name of a class then you have to find code that refers to
> the class and patch it.  If you only care about compiled code, and not
> source code, then class names are dynamic.
>
> -Ralph

This brings up a point that has been bouncing around for a while.  While
Smalltalk sees all things as objects, the one thing that is not currently an
object is source code.  We store the original string from the user as the
master copy, and derive objects from that.  This is the source of the whole
Class name thing, and the need for refactoring them.  If we only kept the
objects (with things like comments intact) and regenerated the source on
need this would not be an issue.  It would also allow richer literals and a
few other things.  It could also support auto-formatting of all source code
if the user chose to do that.  It would require that variable names and
comments be retained in the image, but with modern machines that should not
be a big deal.  Done properly it would also resolve the case where different
intended classes have the same name, as their references would be different.
When editing the source the system could either prompt for desired class,
use the pre-existing reference as a guide, or treat non-unique class names
differently when generating source.

Michael

At 19:00 27.05.2006, Michael wrote:




Hear, hear.

Change the CompiledMethod>>sourcePointers from being indexes into a file to
becoming a link to a source object. What vistas of powerful new IDEs!. Let
the "real thing" be the binary class and method objects as seen by the VM.
Let the programmer see code through an IDE. Carefully isolate the links
between running programs and their sources. Class names, method source, the
notion of "system", persistence,.. will all be features of the IDE. Why
should all parts of an image be programmed the same way?

Cheers,
Trygve



--

Trygve Reenskaug      mailto: [hidden email]
Morgedalsvn. 5A       http://heim.ifi.uio.no/~trygver
N-0378 Oslo           Tel: (+47) 22 49 57 27
Norway

Hi Stef--

 > I have the impression that your Selector is a bit like
 > selectorNamespace of SmallScript? David Simmons introduced a different
 > Symbol equality.

        Probably; I haven't seen the SmallScript implementation.


-C

--
Craig Latta
improvisational musical informaticist
www.netjam.org
Smalltalkers do: [:it | All with: Class, (And love: it)]

Craig Latta skrev:
>
> Hi Ralph--
>
> > I'm just getting into Spoon (hmm, I'm just learning to use Spoon, I'm
> > getting stirred up with Spoon, ... there ought to be a good line
> > here)...
>
>     Yeah, I like "getting a handle on Spoon". :)
Sorry I can't resist:
This time it's the latta that stirs the spoon.

Karl

Hi Craig,

> The latter (as I mentioned previously). And now that you mention
> ProtoObject... :)  Spoon has no need for ProtoObject; in particular,
> Spoon's proxy class works with VM assistance and its superclass can
> be
> anything. I intend to remove ProtoObject.

What does this mean, "I intend to remove ProtoObject?"  Remove it from
what?  Will Object still inherit from ProtoObject in a Spoon system?

If not, how will other proxy's (of the non-Spoon type) work?

thanks..

Hi Chris--

 > > I intend to remove ProtoObject.
 >
 > What does this mean...? Remove it from what?

        Remove it from the minimal object memory.

 > Will Object still inherit from ProtoObject in a Spoon system?

        No, in the minimal Spoon object memory, Object's superclass slot will
refer to nil, and there will be no ProtoObject class.

 > If not, how will other proxies (of the non-Spoon type) work?

        If they need ProtoObject, they will depend on some module that installs
ProtoObject.


        thanks,

-C

--
Craig Latta
improvisational musical informaticist
www.netjam.org
Smalltalkers do: [:it | All with: Class, (And love: it)]

Hi Ralph--

 > > When we encounter an occurrence of "Smalltalk at:", we can find all
 > > the classes in the system which have the given name. We don't have
 > > to use >>allSubclasses since we'll have a cache of all classes in
 > > the system (recall my proposed replacement for the current system
 > > dictionary).
 >
 > I don't understand your replacement for the current system dictionary.
 > It is just an unkeyed collection of classes.  Why bother, when you
 > can already ask Class for its instances, or Object for its subclasses?
 > Why duplicate functionality?

        Oh, I just assumed some uses might want faster access to that bunch of
objects than by trolling the hierarchy; but if there aren't any, then
great, let's discard it. :)

 > Class names are not dynamic if you want source code to work. If you
 > change the name of a class then you have to find code that refers to
 > the class and patch it.

        Right, so you do that. I don't accept that as an argument for class
names not being dynamic. It's like saying that objects with dependents
aren't dynamic. Perhaps I'm missing something.


        thanks,

-C

--
Craig Latta
improvisational musical informaticist
www.netjam.org
Smalltalkers do: [:it | All with: Class, (And love: it)]

Hi Craig, I'm so fascinated by Spoon.

> If they need ProtoObject, they will depend on some module that
> installs
> ProtoObject.

So, if they started from your minimal object memory that would be a
recompile of the entire class hierarchy, right?

For easy-entry into the Spoon and backward compatibility, I hope that a
"public" minimal object memory would include ProtoObject..

 > > If they need ProtoObject, they will depend on some module that
 > > installs ProtoObject.
 >
 > So, if they started from your minimal object memory that would be a
 > recompile of the entire class hierarchy, right?

        Strictly speaking, no, since ProtoObject doesn't add any variables. And
that's convenient, since it means you don't have to load the
ClassBuilder module first. :)  (ClassBuilder is also absent from the
minimal object memory.)

 > For easy-entry into the Spoon and backward compatibility, I hope that
 > a "public" minimal object memory would include ProtoObject...

        Actually, I think very few applications will really need ProtoObject. I
myself will never have a need for it. It's there to support persistence,
which I support in a different way with very small, live object memories
which send remote messages to each other. But nothing will stop people
from making their own starting-point object memories available with
various modules included.

        That's the beauty of taking on the creation of a minimal system: the
decisions about what to leave and what to take out are pretty simple
(take out everything that isn't needed for the system to start and
extend itself). I get to leave it to others how urgent it is to reload
the removed things. Some people will think it's very important to get
Morphic reloaded, others won't, etc.


-C

--
Craig Latta
improvisational musical informaticist
www.netjam.org
Smalltalkers do: [:it | All with: Class, (And love: it)]