Condensed Sources vs. Squeak Maintainence

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
30 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Condensed Sources vs. Squeak Maintainence

Jerome Peace
Hi all,


I've noticed that condensing sources is on the todo
list of the 3dot9 gamma push.

One of the deliverables with 3dot9 final should be a
3.10 with all changes from 3.0.

As one who looks into squeak code for understanding I
have found that the full version history really speeds
tracking down bugs. Often it is the only good way to
discover the programers intent. It is also important
to know how many have mucked in a particular piece of
code. A large number of cooks is one of the smell
tests for troubled code.

So I am very concerned that 3.9 gamma might lose all
this information when the sources get created anew.

It seems to me poor timing to do it at the end of the
development cycle.

A better plan IMHO would be to bring out 3.9 without
compressing the sources.

Then compress sources as the first (and possibly last
step) of a 3.10.

If you choose to do only source compression for 3.10
you would have an adoptable squeak; and a fall back to
3.9 with the full history around for maintainence
sake.

With both 3.9 and 3.10 representing the same point of
development you would not have to do 3.9 full. You
would do 3.10 full. And if you ran into problems you
could check it against 3.9 full and get your insights
that way.

That makes more sense. The numbering even suggest a
major point. (Or you could call the compressed source
version 4.0 and let the spoon version be 5.0).

I believe a great deal of code will rot and squeak
will become a lot more fragile than it already is if
you compress sources now before 3.9 is finalized.

Thank you for your attention to these concerns.

Yours in service, -- Jerome Peace

"Version numbers are cheap. It is patience that is
precious."







__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around
http://mail.yahoo.com 

Reply | Threaded
Open this post in threaded view
|

Re: Condensed Sources vs. Squeak Maintainence

Marcus Denker

On 24.07.2006, at 08:46, Peace Jerome wrote:

> Hi all,
>
>
> I've noticed that condensing sources is on the todo
> list of the 3dot9 gamma push.
>

One of the problems is that the .changes is limited to 32 MB,
and 3.9 7048 is already 25MB, which means people would
very soon see the limit in normal use (e.g. when loading a bunch
of packages).

Another problem is that if we remove the linefeeds in those
couple of hundrets of methods and do a bit of more _ conversion
(which is intended), then we even hit the 32MB limit while
in 3.9alpha.

> One of the deliverables with 3dot9 final should be a
> 3.10 with all changes from 3.0.
>

This is not possible because of two reasons:
        - the MC based process is broken, we can't load from 3.0
          to get 3.9beta. Yes, bad. I am taking full responsibility.
           (We have the complete code history, though. Just not  
easily loadable)
  - The changes file would hit the 32MB limite before you would get
          a complete 39beta even if it would work in theory.

Besides keeping the source, I'm sure you would want to keep
the changeset information, too. Cost: 5MB of image size for 3.9 alone.

So just for a 3.9 with all changes, all in all we are talking about
30MB that nobody needs but system developers. Do we *really*
want to ship that to all the users?

The history mechanism of Squeak does not scale anyway. Why
only ship the history from 3.0? why not the *complete* history? What
do you do in 10 years? 7MB left in the changeset is kind of small...

The whole history mechanism of Squeak does not scale: There
is no need at all to have the old versions on the local disc at all.
What we need is a server that has the complete history that
then can be queried from the clients, everthing else makes
no sense.

For now, we should keep of course the latest beta version with the  
history of
3.9 available. Just that it's not called "3.9", but "3.9 dev" or  
something
like that.

And we should get a student to work on that code history project...

        Marcus


smime.p7s (5K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Condensed Sources vs. Squeak Maintainence

Mathieu SUEN
2006/7/24, Marcus Denker <[hidden email]>:

>
> On 24.07.2006, at 08:46, Peace Jerome wrote:
>
> > Hi all,
> >
> >
> > I've noticed that condensing sources is on the todo
> > list of the 3dot9 gamma push.
> >
>
> One of the problems is that the .changes is limited to 32 MB,
> and 3.9 7048 is already 25MB, which means people would
> very soon see the limit in normal use (e.g. when loading a bunch
> of packages).
>

And can we extend the limit?

Reply | Threaded
Open this post in threaded view
|

Re: Condensed Sources vs. Squeak Maintainence

Marcus Denker

On 24.07.2006, at 09:41, Mathieu SUEN wrote:

> 2006/7/24, Marcus Denker <[hidden email]>:
>>
>> On 24.07.2006, at 08:46, Peace Jerome wrote:
>>
>> > Hi all,
>> >
>> >
>> > I've noticed that condensing sources is on the todo
>> > list of the 3dot9 gamma push.
>> >
>>
>> One of the problems is that the .changes is limited to 32 MB,
>> and 3.9 7048 is already 25MB, which means people would
>> very soon see the limit in normal use (e.g. when loading a bunch
>> of packages).
>>
>
> And can we extend the limit?
>
1) Yes, but it is some work.
2) Unrealistic to do in 3.9beta
3) Even if it is extended, do we then want to ship a 50MB changes
     file? or a 500MB one in 20 years?

For 1): We have properties for CompiledMethods
in 3.9 that could be used to store the sourcePointer as a real  
Integer, as a start.
but then it would be nice to have compiledMethod be "real" objects  
(Tims vm
patch from 1997...). And then the whole thing should be re-thougt  
taking into
account the types of systems we have now. The design is from pre  
1980... there
sure are somet things that can be designed cleaner on todays systems.

E.g. if compressed sources are 5MB. Do we really need to store them  
on Disk?
If yes, why not provide a general mechanism for this that could be  
used for other kinds
of data that is not often accessed (e.g. the Unicode tables, in-image  
font and graphics)?

       Marcus


smime.p7s (5K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Condensed Sources vs. Squeak Maintainence

Mathieu SUEN
2006/7/24, Marcus Denker <[hidden email]>:

>
> On 24.07.2006, at 09:41, Mathieu SUEN wrote:
>
> > 2006/7/24, Marcus Denker <[hidden email]>:
> >>
> >> On 24.07.2006, at 08:46, Peace Jerome wrote:
> >>
> >> > Hi all,
> >> >
> >> >
> >> > I've noticed that condensing sources is on the todo
> >> > list of the 3dot9 gamma push.
> >> >
> >>
> >> One of the problems is that the .changes is limited to 32 MB,
> >> and 3.9 7048 is already 25MB, which means people would
> >> very soon see the limit in normal use (e.g. when loading a bunch
> >> of packages).
> >>
> >
> > And can we extend the limit?
> >
> 1) Yes, but it is some work.
> 2) Unrealistic to do in 3.9beta
> 3) Even if it is extended, do we then want to ship a 50MB changes
>      file? or a 500MB one in 20 years?
>

O yes I see

Reply | Threaded
Open this post in threaded view
|

Re: Condensed Sources vs. Squeak Maintainence

Klaus D. Witzel
In reply to this post by Marcus Denker
Hi Marcus,

since every other project (package) depends on the integrity of the source  
code mechanism, I hereby volunteer to get a new source code subsystem  
(incl. code history) done during this summer. We have no time to wait for  
a new (reliable, scalable, available) source code mechanism, so I intend  
to make resources available for this project.

Here's a draft for a new source code subsystem:

0] (digit for Wolfgang ;-) there are at least three preferences:

1] use the traditional subsystem as is

2] use Dan Ingalls Source Compression as suggested earlier

3] use a service based (network based) mechanism

I've studied the recent additions which where made to CompiledMethod  
w.r.t. MethodProperties, this has to be cleaned up before anything else  
can be done.
        No, I'm not a friend of storing any MetaData *in* the meta data's object.
I propose to put the MethodProperties into the MethodDictionary where they  
belong (same method can *and will* have different properties per class =  
one MethodProperties in each class' method dictionary). As a consequence,  
today's MethodProperties would all be garbage collected (except for the  
few pragmas), because all they store is the selector. And  
CompiledMethod>>#selector is resolved by #methodClass and #keyAtValue: so  
#who can disappear as is intended.

After that, the *new* source code information can be put into the new  
MethodProperties, which allows an easy migration because the old source  
code pointer won't be touched until after migration.

Comments, suggestions, critique, all appreciated.

/Klaus

On Mon, 24 Jul 2006 09:16:58 +0200, Marcus wrote:

>
> On 24.07.2006, at 08:46, Peace Jerome wrote:
>
>> Hi all,
>>
>>
>> I've noticed that condensing sources is on the todo
>> list of the 3dot9 gamma push.
>>
>
> One of the problems is that the .changes is limited to 32 MB,
> and 3.9 7048 is already 25MB, which means people would
> very soon see the limit in normal use (e.g. when loading a bunch
> of packages).
>
> Another problem is that if we remove the linefeeds in those
> couple of hundrets of methods and do a bit of more _ conversion
> (which is intended), then we even hit the 32MB limit while
> in 3.9alpha.
>
>> One of the deliverables with 3dot9 final should be a
>> 3.10 with all changes from 3.0.
>>
>
> This is not possible because of two reasons:
> - the MC based process is broken, we can't load from 3.0
>  to get 3.9beta. Yes, bad. I am taking full responsibility.
>            (We have the complete code history, though. Just not
> easily loadable)
>   - The changes file would hit the 32MB limite before you would get
>  a complete 39beta even if it would work in theory.
>
> Besides keeping the source, I'm sure you would want to keep
> the changeset information, too. Cost: 5MB of image size for 3.9 alone.
>
> So just for a 3.9 with all changes, all in all we are talking about
> 30MB that nobody needs but system developers. Do we *really*
> want to ship that to all the users?
>
> The history mechanism of Squeak does not scale anyway. Why
> only ship the history from 3.0? why not the *complete* history? What
> do you do in 10 years? 7MB left in the changeset is kind of small...
>
> The whole history mechanism of Squeak does not scale: There
> is no need at all to have the old versions on the local disc at all.
> What we need is a server that has the complete history that
> then can be queried from the clients, everthing else makes
> no sense.
>
> For now, we should keep of course the latest beta version with the
> history of
> 3.9 available. Just that it's not called "3.9", but "3.9 dev" or
> something
> like that.
>
> And we should get a student to work on that code history project...
>
>         Marcus



Reply | Threaded
Open this post in threaded view
|

Re: Condensed Sources vs. Squeak Maintainence

Marcus Denker

On 24.07.2006, at 11:54, Klaus D. Witzel wrote:

> Hi Marcus,
>
> since every other project (package) depends on the integrity of the  
> source code mechanism, I hereby volunteer to get a new source code  
> subsystem (incl. code history) done during this summer. We have no  
> time to wait for a new (reliable, scalable, available) source code  
> mechanism, so I intend to make resources available for this project.
>

Good! But we should not wait with 3.9 for this.

> Here's a draft for a new source code subsystem:
>
> 0] (digit for Wolfgang ;-) there are at least three preferences:
>
> 1] use the traditional subsystem as is
>
> 2] use Dan Ingalls Source Compression as suggested earlier
>
> 3] use a service based (network based) mechanism
>
> I've studied the recent additions which where made to  
> CompiledMethod w.r.t. MethodProperties, this has to be cleaned up  
> before anything else can be done.
> No, I'm not a friend of storing any MetaData *in* the meta data's  
> object.
By the selector of a Method is not "meta"! Every method should have a  
name, and if you want a method with another name, make a new method.  
Even the very idea of
different names for the same method is not a good one, as the name is  
hard-coded in the sourcecode, too. It's a direct, non-meta property  
of the method object.

> I propose to put the MethodProperties into the MethodDictionary  
> where they belong (same method can *and will* have different  
> properties per class

Methods should be only installed in one class. yes... Traits, sigh...  
the copying of Traits is evil... I think a better model for traits  
would have been to share Bytecode,
not Methods. But then in Squeak these two concepts have been mangled  
into one for space reasons, which was a bad idea in general. (Or: It  
was a bad idea not
to harvest Tims fix in 1998...).

> = one MethodProperties in each class' method dictionary). As a  
> consequence, today's MethodProperties would all be garbage  
> collected (except for the few pragmas), because all they store is  
> the selector. And CompiledMethod>>#selector is resolved by  
> #methodClass and #keyAtValue: so #who can disappear as is intended.
>
> After that, the *new* source code information can be put into the  
> new MethodProperties, which allows an easy migration because the  
> old source code pointer won't be touched until after migration.
>
You can do that even in the current structure. The sourcePointer is  
now per Method, too.

       Marcus


smime.p7s (5K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Condensed Sources vs. Squeak Maintainence

Edgar J. De Cleene
In reply to this post by Klaus D. Witzel
Klaus D. Witzel puso en su mail :

> 3] use a service based (network based) mechanism
+1

And again I ask if we could have a official class repository from where all
could dowload

Edgar



       
       
               
__________________________________________________
Preguntá. Respondé. Descubrí.
Todo lo que querías saber, y lo que ni imaginabas,
está en Yahoo! Respuestas (Beta).
¡Probalo ya!
http://www.yahoo.com.ar/respuestas


Reply | Threaded
Open this post in threaded view
|

Re: Condensed Sources vs. Squeak Maintainence

Andreas.Raab
In reply to this post by Klaus D. Witzel
 > Comments, suggestions, critique, all appreciated.

Just one: The idea sounds great but can we keep the entirely unrelated
issues of a new source code subsystem and the particular implementation
of per-method properties separate? I don't see why a new source code
subsystem would require a particular implementation of per-method
properties and having the two issues separate allows us to discuss and
assess the merits independently.

Cheers,
   - Andreas


Klaus D. Witzel wrote:

> Hi Marcus,
>
> since every other project (package) depends on the integrity of the
> source code mechanism, I hereby volunteer to get a new source code
> subsystem (incl. code history) done during this summer. We have no time
> to wait for a new (reliable, scalable, available) source code mechanism,
> so I intend to make resources available for this project.
>
> Here's a draft for a new source code subsystem:
>
> 0] (digit for Wolfgang ;-) there are at least three preferences:
>
> 1] use the traditional subsystem as is
>
> 2] use Dan Ingalls Source Compression as suggested earlier
>
> 3] use a service based (network based) mechanism
>
> I've studied the recent additions which where made to CompiledMethod
> w.r.t. MethodProperties, this has to be cleaned up before anything else
> can be done.
>     No, I'm not a friend of storing any MetaData *in* the meta data's
> object.
> I propose to put the MethodProperties into the MethodDictionary where
> they belong (same method can *and will* have different properties per
> class = one MethodProperties in each class' method dictionary). As a
> consequence, today's MethodProperties would all be garbage collected
> (except for the few pragmas), because all they store is the selector.
> And CompiledMethod>>#selector is resolved by #methodClass and
> #keyAtValue: so #who can disappear as is intended.
>
> After that, the *new* source code information can be put into the new
> MethodProperties, which allows an easy migration because the old source
> code pointer won't be touched until after migration.
>
> Comments, suggestions, critique, all appreciated.
>
> /Klaus
>
> On Mon, 24 Jul 2006 09:16:58 +0200, Marcus wrote:
>
>>
>> On 24.07.2006, at 08:46, Peace Jerome wrote:
>>
>>> Hi all,
>>>
>>>
>>> I've noticed that condensing sources is on the todo
>>> list of the 3dot9 gamma push.
>>>
>>
>> One of the problems is that the .changes is limited to 32 MB,
>> and 3.9 7048 is already 25MB, which means people would
>> very soon see the limit in normal use (e.g. when loading a bunch
>> of packages).
>>
>> Another problem is that if we remove the linefeeds in those
>> couple of hundrets of methods and do a bit of more _ conversion
>> (which is intended), then we even hit the 32MB limit while
>> in 3.9alpha.
>>
>>> One of the deliverables with 3dot9 final should be a
>>> 3.10 with all changes from 3.0.
>>>
>>
>> This is not possible because of two reasons:
>>     - the MC based process is broken, we can't load from 3.0
>>       to get 3.9beta. Yes, bad. I am taking full responsibility.
>>            (We have the complete code history, though. Just not
>> easily loadable)
>>       - The changes file would hit the 32MB limite before you would get
>>       a complete 39beta even if it would work in theory.
>>
>> Besides keeping the source, I'm sure you would want to keep
>> the changeset information, too. Cost: 5MB of image size for 3.9 alone.
>>
>> So just for a 3.9 with all changes, all in all we are talking about
>> 30MB that nobody needs but system developers. Do we *really*
>> want to ship that to all the users?
>>
>> The history mechanism of Squeak does not scale anyway. Why
>> only ship the history from 3.0? why not the *complete* history? What
>> do you do in 10 years? 7MB left in the changeset is kind of small...
>>
>> The whole history mechanism of Squeak does not scale: There
>> is no need at all to have the old versions on the local disc at all.
>> What we need is a server that has the complete history that
>> then can be queried from the clients, everthing else makes
>> no sense.
>>
>> For now, we should keep of course the latest beta version with the
>> history of
>> 3.9 available. Just that it's not called "3.9", but "3.9 dev" or
>> something
>> like that.
>>
>> And we should get a student to work on that code history project...
>>
>>         Marcus
>
>
>
>


Reply | Threaded
Open this post in threaded view
|

Re: Condensed Sources vs. Squeak Maintainence

Klaus D. Witzel
In reply to this post by Marcus Denker
Hi Marcus,

on Mon, 24 Jul 2006 12:11:31 +0200, you wrote:

> On 24.07.2006, at 11:54, Klaus D. Witzel wrote:
>
>> Hi Marcus,
>>
>> since every other project (package) depends on the integrity of the
>> source code mechanism, I hereby volunteer to get a new source code
>> subsystem (incl. code history) done during this summer. We have no
>> time to wait for a new (reliable, scalable, available) source code
>> mechanism, so I intend to make resources available for this project.
>>
>
> Good! But we should not wait with 3.9 for this.

O.K.

>> Here's a draft for a new source code subsystem:
>>
>> 0] (digit for Wolfgang ;-) there are at least three preferences:
>>
>> 1] use the traditional subsystem as is
>>
>> 2] use Dan Ingalls Source Compression as suggested earlier
>>
>> 3] use a service based (network based) mechanism
>>
>> I've studied the recent additions which where made to
>> CompiledMethod w.r.t. MethodProperties, this has to be cleaned up
>> before anything else can be done.
>> No, I'm not a friend of storing any MetaData *in* the meta data's
>> object.
>
> By the selector of a Method is not "meta"!

I'm sorry it is "meta" by tradition. Otherwise it would have been stored  
in the method (literals) since the 80's of the previous century. And if  
you cannot agree with that view:

only the VM knows about the relationship between the selector used for  
method lookup and the found and executed method. This *is* what meta is  
about, that's what I think.

> Every method should have a
> name, and if you want a method with another name, make a new method.

No clash here: if I throw the MethodProperties oop out of CompiledMethod's  
literals then I can still fulfill this your requirement (and you owe me a  
beer ;-).

> Even the very idea of
> different names for the same method is not a good one, as the name is
> hard-coded in the sourcecode, too. It's a direct, non-meta property
> of the method object.

Sorry (same argument as above). It is, and I have to agree, a  
*substantial* part of the source-to-compiled method messenger "RNA"  
enzymatic process. But the result cannot tell which name its gene had for  
it (we human beings can, same for source/compiled code, to some extent).  
Take as a counter example #DoItwith:with:... and the cause for having  
CompiledMethod>>defaultSelector.

But anyways, I do not see a clash here and it's only a matter  
implementation.

Having said that, was is really missing is " an  
ObsoleteCompiledMethod(4711) ", mantained by MethodDictionary with key :=  
oldMethod and value := aClass -> aSelector. A new global variable  
ObsoleteCompiledMethods can hold, with advantage, an instance of  
WeakIdentityKeyDictionary for these keys and values.

>> I propose to put the MethodProperties into the MethodDictionary
>> where they belong (same method can *and will* have different
>> properties per class
>
> Methods should be only installed in one class.

And the system should be transparent if not so. But again, it can be  
implemented the way you describe.

> yes... Traits, sigh...
> the copying of Traits is evil... I think a better model for traits
> would have been to share Bytecode,
> not Methods. But then in Squeak these two concepts have been mangled
> into one for space reasons, which was a bad idea in general. (Or: It
> was a bad idea not
> to harvest Tims fix in 1998...).

I have no problem with this. This will (perhaps) be reviewed once people  
take advantage of Squeak Trait(s).

>> = one MethodProperties in each class' method dictionary). As a
>> consequence, today's MethodProperties would all be garbage
>> collected (except for the few pragmas), because all they store is
>> the selector. And CompiledMethod>>#selector is resolved by
>> #methodClass and #keyAtValue: so #who can disappear as is intended.
>>
>> After that, the *new* source code information can be put into the
>> new MethodProperties, which allows an easy migration because the
>> old source code pointer won't be touched until after migration.
>>
>
> You can do that even in the current structure. The sourcePointer is
> now per Method, too.

? I was referring to the old (as is now) and, at the same time, during  
migration, to the then new sourcePointer, with emphasis on having both  
available during migration. What was it what you meant?

/Klaus

>
>        Marcus



Reply | Threaded
Open this post in threaded view
|

Re: Condensed Sources vs. Squeak Maintainence

Klaus D. Witzel
In reply to this post by Andreas.Raab
On Mon, 24 Jul 2006 14:00:16 +0200, Andreas Raab wrote:

>  > Comments, suggestions, critique, all appreciated.
>
> Just one: The idea sounds great but can we keep the entirely unrelated  
> issues of a new source code subsystem and the particular implementation  
> of per-method properties separate? I don't see why a new source code  
> subsystem would require a particular implementation of per-method  
> properties and having the two issues separate allows us to discuss and  
> assess the merits independently.

You're right, and thank you for pointing me to the independence of the two  
issues. As everybody else in software industry, I was suggesting a  
particular implementation (of new sourcePointer) instead of a sound and  
complete design ;-)

/Klaus

> Cheers,
>    - Andreas


Reply | Threaded
Open this post in threaded view
|

Re: Condensed Sources vs. Squeak Maintainence

Marcus Denker
In reply to this post by Klaus D. Witzel

On 24.07.2006, at 18:21, Klaus D. Witzel wrote:


>>
>> By the selector of a Method is not "meta"!
>
> I'm sorry it is "meta" by tradition. Otherwise it would have been  
> stored in the method (literals) since the 80's of the previous  
> century. And if you cannot agree with that view:
>
> only the VM knows about the relationship between the selector used  
> for method lookup and the found and executed method. This *is* what  
> meta is about, that's what I think.
>
If the model that we want to have is just "execution", then this is  
correct. This is what we have now: A model describing classes/
methods, where the description of the
classes is both for the machine and the programmer, but for methods  
we only model execution. What I would like to see (now that machines  
have become so
much more powerful) is to move this model up one level and make it's  
main goal to be a model of the program for engineering, while keeping  
this model executionable.

This is why a method should know it's class (this is what a developer  
thinks about a method) and it's name... the whole mode of the system  
should be on an abstraction
level of  that kind of model that you would design for a re-
engineering or visualisation tool...

e.g. for execution, a class needs no name, just like a method needs  
no name. You could have nameless "CompiledClasses". But the objects  
encoding classes we do
have a slot for the name... Because they have not only the  
interpreter as the client, but the developer tools (and the  
developer), too. Because it makes no sense to
destinguish between the two: The higher level model with name is  
perfectly executable, too.

It makes the tools much simpler: In 3.9, methods know their class  
without having to search the whole system. As soon as that was done,  
I started to refactor a little... it
turns out that this information makes the design simpler: e.g no need  
to call "decompileClass: aClass selector: selector" on the  
compiledMethod... just call
decompile on the method, no need to funnel the extra information to  
all the places where they are needed. For the tools, it feels much  
more natural if the methods
know more than nothing... as then the class/method structure will be  
your meta model that you can use directly.

      Marcus
 


smime.p7s (5K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Condensed Sources vs. Squeak Maintainence

Klaus D. Witzel
On Mon, 24 Jul 2006 19:44:01 +0200, Marcus Denker wrote:

> On 24.07.2006, at 18:21, Klaus D. Witzel wrote:
>
>
>>>
>>> By the selector of a Method is not "meta"!
>>
>> I'm sorry it is "meta" by tradition. Otherwise it would have been
>> stored in the method (literals) since the 80's of the previous
>> century. And if you cannot agree with that view:
>>
>> only the VM knows about the relationship between the selector used
>> for method lookup and the found and executed method. This *is* what
>> meta is about, that's what I think.
>>
>
> If the model that we want to have is just "execution", then this is
> correct. This is what we have now: A model describing classes/
> methods, where the description of the
> classes is both for the machine and the programmer, but for methods
> we only model execution. What I would like to see (now that machines
> have become so
> much more powerful) is to move this model up one level and make it's
> main goal to be a model of the program for engineering, while keeping
> this model executionable.

I'm in sync.

> This is why a method should know it's class (this is what a developer
> thinks about a method) and it's name... the whole mode of the system
> should be on an abstraction
> level of  that kind of model that you would design for a re-
> engineering or visualisation tool...

But by using methodClass (to which I fully agree), every method knows it  
class, which knows its method dictionary, which knows the corresponding  
selector (all a matter of implementation). No need for MethodProperties  
here.

And yes (and to make it clear): every method nowadays has methodClass  
(except perhaps those circumscribed in a comment, similiar to 'bad case'  
in the other thread). Since any two ~= classes cannot be the same type, so  
their methods. So methodClass *must* be checked for by CompiledMethod>>#=  
... (which, BTW, doesn't check the selector, in any version I've seen so  
far ;-)

> e.g. for execution, a class needs no name, just like a method needs
> no name. You could have nameless "CompiledClasses". But the objects
> encoding classes we do
> have a slot for the name...

Good point. This is equivalent to (Smalltalk keyAtValue: aSlot), same as  
for (aMethodDictionary keyAtValue: aCompiledMethod), a matter of  
performance at the time of invention (CompiledMethod instanceCount > (...  
allObjectsdo: ... isBehavior ...) size, a subject for lengthy discussions  
in modern times ;-)

> Because they have not only the
> interpreter as the client, but the developer tools (and the
> developer), too. Because it makes no sense to
> destinguish between the two: The higher level model with name is
> perfectly executable, too.

Sure, the higher level is "just" about *organization*, *reuse* and other  
elaborate concepts, a typical reason for introducing execution unrelated  
meta data (similiar to ClassDescription).

> It makes the tools much simpler: In 3.9, methods know their class
> without having to search the whole system. As soon as that was done,
> I started to refactor a little... it
> turns out that this information makes the design simpler: e.g no need
> to call "decompileClass: aClass selector: selector" on the
> compiledMethod... just call
> decompile on the method, no need to funnel the extra information to
> all the places where they are needed.

Yes, when I saw that it was quite a surprise how easy things can be ;-)  
Good reason for marking these oldies with #deprecated:.

> For the tools, it feels much
> more natural if the methods
> know more than nothing... as then the class/method structure will be
> your meta model that you can use directly.

O.K. but the level we are talking about here is equivalent to [at least]  
Behavior>>#allCallsOn: why the heck should Behavior know anything about  
calls to symbols in CompiledMethods? It delegates.

So can CompiledMethod (delegate) when asked the queries you mentioned. It  
doesn't need to store anything else than Blue Book's methodClass for  
obtaining these answers (by delegation).

I second implementing all possible queries and oops related to  
CompiledMethod in CompiledMethod. Perhaps as a separator for a redesign:  
oops which are unrelated to CompiledMethod>>#= (unrelated to type-,  
function-, computational-equivalency "PCE") do not belong to  
CompiledMethod. A few examples for the unrelated: #categoryOfElement:,  
#tempNames, source-origin-server, source-origin-path, source-origin-uri,  
source-origin-version, ..., source-origin-name (selector) they shouted  
when the baby was born ;-)

/Klaus

>       Marcus



Reply | Threaded
Open this post in threaded view
|

Re: Condensed Sources vs. Squeak Maintainence

stéphane ducasse-2
In reply to this post by Marcus Denker
>
>> One of the deliverables with 3dot9 final should be a
>> 3.10 with all changes from 3.0.
>>
>
> This is not possible because of two reasons:
> - the MC based process is broken, we can't load from 3.0
>  to get 3.9beta. Yes, bad. I am taking full responsibility.

I'm too.
I invite everybody to build something and we will use it.

>           (We have the complete code history, though. Just not  
> easily loadable)
>   - The changes file would hit the 32MB limite before you would get
>  a complete 39beta even if it would work in theory.
>
> Besides keeping the source, I'm sure you would want to keep
> the changeset information, too. Cost: 5MB of image size for 3.9 alone.
>
> So just for a 3.9 with all changes, all in all we are talking about
> 30MB that nobody needs but system developers. Do we *really*
> want to ship that to all the users?
>
> The history mechanism of Squeak does not scale anyway. Why
> only ship the history from 3.0? why not the *complete* history? What
> do you do in 10 years? 7MB left in the changeset is kind of small...
>
> The whole history mechanism of Squeak does not scale: There
> is no need at all to have the old versions on the local disc at all.
> What we need is a server that has the complete history that
> then can be queried from the clients, everthing else makes
> no sense.
>
> For now, we should keep of course the latest beta version with the  
> history of
> 3.9 available. Just that it's not called "3.9", but "3.9 dev" or  
> something
> like that.
>
> And we should get a student to work on that code history project...
>
>        Marcus


Reply | Threaded
Open this post in threaded view
|

Re: Condensed Sources vs. Squeak Maintainence

stéphane ducasse-2
In reply to this post by Andreas.Raab
+ 1
Changes are necessary but costly so we should go stepwise.

Stef

> > Comments, suggestions, critique, all appreciated.
>
> Just one: The idea sounds great but can we keep the entirely  
> unrelated issues of a new source code subsystem and the particular  
> implementation of per-method properties separate? I don't see why a  
> new source code subsystem would require a particular implementation  
> of per-method properties and having the two issues separate allows  
> us to discuss and assess the merits independently.
>
> Cheers,
>   - Andreas
>
>
> Klaus D. Witzel wrote:
>> Hi Marcus,
>> since every other project (package) depends on the integrity of  
>> the source code mechanism, I hereby volunteer to get a new source  
>> code subsystem (incl. code history) done during this summer. We  
>> have no time to wait for a new (reliable, scalable, available)  
>> source code mechanism, so I intend to make resources available for  
>> this project.
>> Here's a draft for a new source code subsystem:
>> 0] (digit for Wolfgang ;-) there are at least three preferences:
>> 1] use the traditional subsystem as is
>> 2] use Dan Ingalls Source Compression as suggested earlier
>> 3] use a service based (network based) mechanism
>> I've studied the recent additions which where made to  
>> CompiledMethod w.r.t. MethodProperties, this has to be cleaned up  
>> before anything else can be done.
>>     No, I'm not a friend of storing any MetaData *in* the meta  
>> data's object.
>> I propose to put the MethodProperties into the MethodDictionary  
>> where they belong (same method can *and will* have different  
>> properties per class = one MethodProperties in each class' method  
>> dictionary). As a consequence, today's MethodProperties would all  
>> be garbage collected (except for the few pragmas), because all  
>> they store is the selector. And CompiledMethod>>#selector is  
>> resolved by #methodClass and #keyAtValue: so #who can disappear as  
>> is intended.
>> After that, the *new* source code information can be put into the  
>> new MethodProperties, which allows an easy migration because the  
>> old source code pointer won't be touched until after migration.
>> Comments, suggestions, critique, all appreciated.
>> /Klaus
>> On Mon, 24 Jul 2006 09:16:58 +0200, Marcus wrote:
>>>
>>> On 24.07.2006, at 08:46, Peace Jerome wrote:
>>>
>>>> Hi all,
>>>>
>>>>
>>>> I've noticed that condensing sources is on the todo
>>>> list of the 3dot9 gamma push.
>>>>
>>>
>>> One of the problems is that the .changes is limited to 32 MB,
>>> and 3.9 7048 is already 25MB, which means people would
>>> very soon see the limit in normal use (e.g. when loading a bunch
>>> of packages).
>>>
>>> Another problem is that if we remove the linefeeds in those
>>> couple of hundrets of methods and do a bit of more _ conversion
>>> (which is intended), then we even hit the 32MB limit while
>>> in 3.9alpha.
>>>
>>>> One of the deliverables with 3dot9 final should be a
>>>> 3.10 with all changes from 3.0.
>>>>
>>>
>>> This is not possible because of two reasons:
>>>     - the MC based process is broken, we can't load from 3.0
>>>       to get 3.9beta. Yes, bad. I am taking full responsibility.
>>>            (We have the complete code history, though. Just not
>>> easily loadable)
>>>       - The changes file would hit the 32MB limite before you  
>>> would get
>>>       a complete 39beta even if it would work in theory.
>>>
>>> Besides keeping the source, I'm sure you would want to keep
>>> the changeset information, too. Cost: 5MB of image size for 3.9  
>>> alone.
>>>
>>> So just for a 3.9 with all changes, all in all we are talking about
>>> 30MB that nobody needs but system developers. Do we *really*
>>> want to ship that to all the users?
>>>
>>> The history mechanism of Squeak does not scale anyway. Why
>>> only ship the history from 3.0? why not the *complete* history? What
>>> do you do in 10 years? 7MB left in the changeset is kind of small...
>>>
>>> The whole history mechanism of Squeak does not scale: There
>>> is no need at all to have the old versions on the local disc at all.
>>> What we need is a server that has the complete history that
>>> then can be queried from the clients, everthing else makes
>>> no sense.
>>>
>>> For now, we should keep of course the latest beta version with the
>>> history of
>>> 3.9 available. Just that it's not called "3.9", but "3.9 dev" or
>>> something
>>> like that.
>>>
>>> And we should get a student to work on that code history project...
>>>
>>>         Marcus
>
>


Reply | Threaded
Open this post in threaded view
|

Re: Condensed Sources vs. Squeak Maintainence

Andreas.Raab
In reply to this post by Klaus D. Witzel
Klaus D. Witzel wrote:
> So can CompiledMethod (delegate) when asked the queries you mentioned.
> It doesn't need to store anything else than Blue Book's methodClass for
> obtaining these answers (by delegation).

Not quite true. It is easy to see that in a situation like here:

   Object compile:'foo [42]'.
   fooBlock := Object new foo.
   Object compile:'foo ^21'.

you can't (without major modifications of classes) determine the
properties of "fooBlock home" (say, for source retrieval). This is a
relevant pattern as I've found in the past when dealing with changes to
compiled methods - while not exactly common it is also not rare enough
that it can be easily ignored.

Cheers,
   - Andreas



Reply | Threaded
Open this post in threaded view
|

Still about methodClass [was: Condensed Sources vs. Squeak Maintainence]

Klaus D. Witzel
On Tue, 25 Jul 2006 08:53:49 +0200, Andreas Raab wrote:

> Klaus D. Witzel wrote:
>> So can CompiledMethod (delegate) when asked the queries you mentioned.  
>> It doesn't need to store anything else than Blue Book's methodClass for  
>> obtaining these answers (by delegation).
>
> Not quite true. It is easy to see that in a situation like here:
>
>    Object compile:'foo [42]'.
>    fooBlock := Object new foo.
>    Object compile:'foo ^21'.
>
> you can't (without major modifications of classes) determine the  
> properties of "fooBlock home" (say, for source retrieval).

This one (and recompiles while in debugger, fileIn's, another browser,  
parallel to a forked process, etc) can easily be addressed by the new  
ObsoleteCompiledMethods dictionary which I mentioned earlier. No major  
change (no change at all :) to classes necessary, just a proper action  
during aMethodDictionary xyzAt:ifAbsent:.

I could give more [boring] details if that's wanted.

> This is a relevant pattern as I've found in the past when dealing with  
> changes to compiled methods - while not exactly common it is also not  
> rare enough that it can be easily ignored.

Yes, neglection of these would have to be categorized under  
'major-failure'.

/Klaus

> Cheers,
>    - Andreas
>
>
>
>



Reply | Threaded
Open this post in threaded view
|

Re: Still about methodClass [was: Condensed Sources vs. Squeak Maintainence]

Andreas.Raab
Klaus D. Witzel wrote:
> I could give more [boring] details if that's wanted.

Actually, if you want my attention, talk about the new source code
subsystem ;-)

The method properties discussion is a red herring in my eyes - we have a
functioning version today and if that needs changing or not is in the
eye of the beholder. Personally (having done it both ways) I strongly
prefer the current solution since it has some nice properties (like that
all relevant attributes are reachable via direct references from the
method and therefore garbage collection will do The Right Thing without
additional help; that a generic copying mechanism doesn't have to know
that some vital parts of compiled methods are stored elsewhere; that
extensions can be done in one place rather than many) but things can
certainly be done differently. Discussing those merits would require
some more data points other than saying "yes, it can be done" (because I
know it can, I have done it before ;-) and would likely mean some fairly
subjective comparisons of what is "nicer" or "more understandable" or
somesuch.

In light of that, I'd rather discuss an area that actually needs fixing,
like the source code subsystem.

Cheers,
   - Andreas

Reply | Threaded
Open this post in threaded view
|

Re: Still about methodClass

Klaus D. Witzel
On Tue, 25 Jul 2006 17:19:11 +0200, Andreas Raab wrote:

> Klaus D. Witzel wrote:
>> I could give more [boring] details if that's wanted.
>
> Actually, if you want my attention, talk about the new source code  
> subsystem ;-)

So we'll take branch here, the new source code subsystem topic will get a  
new thread.

> The method properties discussion is a red herring in my eyes - we have a  
> functioning version today and if that needs changing or not is in the  
> eye of the beholder. Personally (having done it both ways) I strongly  
> prefer the current solution since it has some nice properties (like that  
> all relevant attributes are reachable via direct references from the  
> method and therefore garbage collection will do The Right Thing without  
> additional help; that a generic copying mechanism doesn't have to know  
> that some vital parts of compiled methods are stored elsewhere; that  
> extensions can be done in one place rather than many) but things can  
> certainly be done differently. Discussing those merits would require  
> some more data points other than saying "yes, it can be done" (because I  
> know it can, I have done it before ;-) and would likely mean some fairly  
> subjective comparisons of what is "nicer" or "more understandable" or  
> somesuch.

Good points. May I stress that my concern is the *current* *integrity* of

(nil systemNavigation
        allMethodsSelect: [:method |
                method hasNewPropertyFormat == false]) size
=> 0
       
|x y | x := y := 0.
nil systemNavigation
        allMethodsSelect: [:method |
                method literals last isVariableBinding
                ifTrue: [x := 1 + x]
                ifFalse: [y := 1 + y]. false].
x @ y => 49487@0
       
|x y | x := y := 0.
CompiledMethod allInstancesDo: [:z |
        z literals last isVariableBinding
                ifTrue: [x := 1 + x]
                ifFalse: [y := 1 + y]].
x@y => 49035@5

(note the discrepancy, 49487 versus 49035, 0 versus 5, relative to  
3.9#7048)

As discussed with Marcus last night (1st Squeak Stammtisch in Bern, 4  
Squeakers :) I'm no longer in opposition to how and where MethodProperties  
are stored and maintained. And if I understand your comment above  
correctly, that item is hereby closed.

But aCompiledMethod methodClass (aMethod literals last isVariableBinding)  
must be solved (and perhaps the related implementation must be reviewed)  
before I'm interested in going a step further into the direction of a new  
source code subsystem. My intention here is that there is consensus on WHO  
exactly tells which class a method belongs to; when can (if at all) that  
change; how are obsolete methods managed in the future. This must be as  
reliable as possible, otherwise we can forget about aCompiledMethod>>#=  
for use in any other (sub)system. No, it's not my intention to invent  
something which is ersatz (<= WordNet/2.1) for aCompiledMethod>>#=.

A nice and bad (perhaps no *that* close but nevertheless frightening)  
example is ChangeRecord>>#methodClass, it asks Smalltalk (regardless of  
obsolence)...

In other words and in a single statement: clean up the #methodClass mess  
in the image.

Thereafter we can introduce (and/or reuse) concepts which support a source  
code view on current, obsolete, historized and future CompiledMethods, a'  
la distributed authoring and versioning.

Please nobody misunderstand me here: propose a fix for #methodClass  
(obsolete methods as well) and we can immediately turn over to the more  
interesting things like managing the future (so to speak).

> In light of that, I'd rather discuss an area that actually needs fixing,  
> like the source code subsystem.

Me too; let's fix first things first.

/Klaus

> Cheers,
>    - Andreas
>
>



Reply | Threaded
Open this post in threaded view
|

Re: Still about methodClass

Marcus Denker

On 26.07.2006, at 07:36, Klaus D. Witzel wrote:

> On Tue, 25 Jul 2006 17:19:11 +0200, Andreas Raab wrote:
>
>> Klaus D. Witzel wrote:
>>> I could give more [boring] details if that's wanted.
>>
>> Actually, if you want my attention, talk about the new source code  
>> subsystem ;-)
>
> So we'll take branch here, the new source code subsystem topic will  
> get a new thread.
>
>> The method properties discussion is a red herring in my eyes - we  
>> have a functioning version today and if that needs changing or not  
>> is in the eye of the beholder. Personally (having done it both  
>> ways) I strongly prefer the current solution since it has some  
>> nice properties (like that all relevant attributes are reachable  
>> via direct references from the method and therefore garbage  
>> collection will do The Right Thing without additional help; that a  
>> generic copying mechanism doesn't have to know that some vital  
>> parts of compiled methods are stored elsewhere; that extensions  
>> can be done in one place rather than many) but things can  
>> certainly be done differently. Discussing those merits would  
>> require some more data points other than saying "yes, it can be  
>> done" (because I know it can, I have done it before ;-) and would  
>> likely mean some fairly subjective comparisons of what is "nicer"  
>> or "more understandable" or somesuch.
>
> Good points. May I stress that my concern is the *current*  
> *integrity* of
>
> (nil systemNavigation
> allMethodsSelect: [:method |
> method hasNewPropertyFormat == false]) size
> => 0
All methods have been recompiled to use the new format (with class  
binding in last literal)

>
> |x y | x := y := 0.
> nil systemNavigation
> allMethodsSelect: [:method |
> method literals last isVariableBinding
> ifTrue: [x := 1 + x]
> ifFalse: [y := 1 + y]. false].
> x @ y => 49487@0
>
> |x y | x := y := 0.
> CompiledMethod allInstancesDo: [:z |
> z literals last isVariableBinding
> ifTrue: [x := 1 + x]
> ifFalse: [y := 1 + y]].
> x@y => 49035@5
>
> (note the discrepancy, 49487 versus 49035,
non-locally intalled traits methods.

> 0 versus 5, relative to 3.9#7048)
>

There are some methods on the stack that have not yet finished  
executing... should be fixed.
  It's on the never-ending list of stuff on the todo.

> As discussed with Marcus last night (1st Squeak Stammtisch in Bern,  
> 4 Squeakers :) I'm no longer in opposition to how and where  
> MethodProperties are stored and maintained. And if I understand  
> your comment above correctly, that item is hereby closed.
>
> But aCompiledMethod methodClass (aMethod literals last  
> isVariableBinding) must be solved (and perhaps the related  
> implementation must be reviewed) before I'm interested in going a  
> step further into the direction of a new source code subsystem. My  
> intention here is that there is consensus on WHO exactly tells  
> which class a method belongs to; when can (if at all) that change;  
> how are obsolete methods managed in the future. This must be as  
> reliable as possible, otherwise we can forget about  
> aCompiledMethod>>#= for use in any other (sub)system. No, it's not  
> my intention to invent something which is ersatz (<= WordNet/2.1)  
> for aCompiledMethod>>#=.
>
#= does not take the Class of the method into account, as any client  
for equality would not want that. You want to ask "which methods are  
the same in the system". If we
take the Class into account, you will never find any equeal methods...

> A nice and bad (perhaps no *that* close but nevertheless  
> frightening) example is ChangeRecord>>#methodClass, it asks  
> Smalltalk (regardless of obsolence)...
> words and in a single statement: clean up the #methodClass mess in  
> the image.
>

The whole fileOut / Changes stuff os a huge huge mess. But  
#methodClass in ChangeRecord has not much to do with #methodClass on  
CompiledMethod...

Seriously, the only way to improve the situation is to improve it...  
we had many people telling that only perfect solutions are  
acceptable. Or worthy of Squeak
or something. This is wrong: We need *better* solutions, not perfect  
ones. If we would have just made everything continously a little bit  
better over the last
10 years, we would have quite a cool system now.

The idea of searching for the "perfect" solution only (while letting  
everything rot) is the reason for a lot of what we see now.

        Marcus







smime.p7s (5K) Download Attachment
12