Smalltalk › Squeak › Squeak - Dev

Condensed Sources vs. Squeak Maintainence

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

30 messages Options

Jerome Peace

Condensed Sources vs. Squeak Maintainence

Hi all,

I've noticed that condensing sources is on the todo
list of the 3dot9 gamma push.

One of the deliverables with 3dot9 final should be a
3.10 with all changes from 3.0.

As one who looks into squeak code for understanding I
have found that the full version history really speeds
tracking down bugs. Often it is the only good way to
discover the programers intent. It is also important
to know how many have mucked in a particular piece of
code. A large number of cooks is one of the smell
tests for troubled code.

So I am very concerned that 3.9 gamma might lose all
this information when the sources get created anew.

It seems to me poor timing to do it at the end of the
development cycle.

A better plan IMHO would be to bring out 3.9 without
compressing the sources.

Then compress sources as the first (and possibly last
step) of a 3.10.

If you choose to do only source compression for 3.10
you would have an adoptable squeak; and a fall back to
3.9 with the full history around for maintainence
sake.

With both 3.9 and 3.10 representing the same point of
development you would not have to do 3.9 full. You
would do 3.10 full. And if you ran into problems you
could check it against 3.9 full and get your insights
that way.

That makes more sense. The numbering even suggest a
major point. (Or you could call the compressed source
version 4.0 and let the spoon version be 5.0).

I believe a great deal of code will rot and squeak
will become a lot more fragile than it already is if
you compress sources now before 3.9 is finalized.

Thank you for your attention to these concerns.

Yours in service, -- Jerome Peace

"Version numbers are cheap. It is patience that is
precious."

__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com

Marcus Denker

Re: Condensed Sources vs. Squeak Maintainence

On 24.07.2006, at 08:46, Peace Jerome wrote:

> Hi all,
>
>
> I've noticed that condensing sources is on the todo
> list of the 3dot9 gamma push.
>

One of the problems is that the .changes is limited to 32 MB,
and 3.9 7048 is already 25MB, which means people would
very soon see the limit in normal use (e.g. when loading a bunch
of packages).

Another problem is that if we remove the linefeeds in those
couple of hundrets of methods and do a bit of more _ conversion
(which is intended), then we even hit the 32MB limit while
in 3.9alpha.

> One of the deliverables with 3dot9 final should be a
> 3.10 with all changes from 3.0.
>

This is not possible because of two reasons:
- the MC based process is broken, we can't load from 3.0
to get 3.9beta. Yes, bad. I am taking full responsibility.
(We have the complete code history, though. Just not
easily loadable)
- The changes file would hit the 32MB limite before you would get
a complete 39beta even if it would work in theory.

Besides keeping the source, I'm sure you would want to keep
the changeset information, too. Cost: 5MB of image size for 3.9 alone.

So just for a 3.9 with all changes, all in all we are talking about
30MB that nobody needs but system developers. Do we *really*
want to ship that to all the users?

The history mechanism of Squeak does not scale anyway. Why
only ship the history from 3.0? why not the *complete* history? What
do you do in 10 years? 7MB left in the changeset is kind of small...

The whole history mechanism of Squeak does not scale: There
is no need at all to have the old versions on the local disc at all.
What we need is a server that has the complete history that
then can be queried from the clients, everthing else makes
no sense.

For now, we should keep of course the latest beta version with the
history of
3.9 available. Just that it's not called "3.9", but "3.9 dev" or
something
like that.

And we should get a student to work on that code history project...

Marcus

smime.p7s (5K) Download Attachment

Mathieu SUEN

Re: Condensed Sources vs. Squeak Maintainence

2006/7/24, Marcus Denker <[hidden email]>:

And can we extend the limit?

Marcus Denker

Re: Condensed Sources vs. Squeak Maintainence

On 24.07.2006, at 09:41, Mathieu SUEN wrote:

> 2006/7/24, Marcus Denker <[hidden email]>:
>>
>> On 24.07.2006, at 08:46, Peace Jerome wrote:
>>
>> > Hi all,
>> >
>> >
>> > I've noticed that condensing sources is on the todo
>> > list of the 3dot9 gamma push.
>> >
>>
>> One of the problems is that the .changes is limited to 32 MB,
>> and 3.9 7048 is already 25MB, which means people would
>> very soon see the limit in normal use (e.g. when loading a bunch
>> of packages).
>>
>
> And can we extend the limit?
>

1) Yes, but it is some work.

2) Unrealistic to do in 3.9beta
3) Even if it is extended, do we then want to ship a 50MB changes
file? or a 500MB one in 20 years?

For 1): We have properties for CompiledMethods
in 3.9 that could be used to store the sourcePointer as a real
Integer, as a start.
but then it would be nice to have compiledMethod be "real" objects
(Tims vm
patch from 1997...). And then the whole thing should be re-thougt
taking into
account the types of systems we have now. The design is from pre
1980... there
sure are somet things that can be designed cleaner on todays systems.

E.g. if compressed sources are 5MB. Do we really need to store them
on Disk?
If yes, why not provide a general mechanism for this that could be
used for other kinds
of data that is not often accessed (e.g. the Unicode tables, in-image
font and graphics)?

Marcus

smime.p7s (5K) Download Attachment

Mathieu SUEN

Re: Condensed Sources vs. Squeak Maintainence

2006/7/24, Marcus Denker <[hidden email]>:

>
> On 24.07.2006, at 09:41, Mathieu SUEN wrote:
>
> > 2006/7/24, Marcus Denker <[hidden email]>:
> >>
> >> On 24.07.2006, at 08:46, Peace Jerome wrote:
> >>
> >> > Hi all,
> >> >
> >> >
> >> > I've noticed that condensing sources is on the todo
> >> > list of the 3dot9 gamma push.
> >> >
> >>
> >> One of the problems is that the .changes is limited to 32 MB,
> >> and 3.9 7048 is already 25MB, which means people would
> >> very soon see the limit in normal use (e.g. when loading a bunch
> >> of packages).
> >>
> >
> > And can we extend the limit?
> >
> 1) Yes, but it is some work.
> 2) Unrealistic to do in 3.9beta
> 3) Even if it is extended, do we then want to ship a 50MB changes
> file? or a 500MB one in 20 years?
>

O yes I see

Klaus D. Witzel

Re: Condensed Sources vs. Squeak Maintainence

In reply to this post by Marcus Denker

Hi Marcus,

since every other project (package) depends on the integrity of the source
code mechanism, I hereby volunteer to get a new source code subsystem
(incl. code history) done during this summer. We have no time to wait for
a new (reliable, scalable, available) source code mechanism, so I intend
to make resources available for this project.

Here's a draft for a new source code subsystem:

0] (digit for Wolfgang ;-) there are at least three preferences:

1] use the traditional subsystem as is

2] use Dan Ingalls Source Compression as suggested earlier

3] use a service based (network based) mechanism

I've studied the recent additions which where made to CompiledMethod
w.r.t. MethodProperties, this has to be cleaned up before anything else
can be done.
No, I'm not a friend of storing any MetaData *in* the meta data's object.
I propose to put the MethodProperties into the MethodDictionary where they
belong (same method can *and will* have different properties per class =
one MethodProperties in each class' method dictionary). As a consequence,
today's MethodProperties would all be garbage collected (except for the
few pragmas), because all they store is the selector. And
CompiledMethod>>#selector is resolved by #methodClass and #keyAtValue: so
#who can disappear as is intended.

After that, the *new* source code information can be put into the new
MethodProperties, which allows an easy migration because the old source
code pointer won't be touched until after migration.

Comments, suggestions, critique, all appreciated.

/Klaus

On Mon, 24 Jul 2006 09:16:58 +0200, Marcus wrote:

>
> On 24.07.2006, at 08:46, Peace Jerome wrote:
>
>> Hi all,
>>
>>
>> I've noticed that condensing sources is on the todo
>> list of the 3dot9 gamma push.
>>
>
> One of the problems is that the .changes is limited to 32 MB,
> and 3.9 7048 is already 25MB, which means people would
> very soon see the limit in normal use (e.g. when loading a bunch
> of packages).
>
> Another problem is that if we remove the linefeeds in those
> couple of hundrets of methods and do a bit of more _ conversion
> (which is intended), then we even hit the 32MB limit while
> in 3.9alpha.
>
>> One of the deliverables with 3dot9 final should be a
>> 3.10 with all changes from 3.0.
>>
>
> This is not possible because of two reasons:
> - the MC based process is broken, we can't load from 3.0
> to get 3.9beta. Yes, bad. I am taking full responsibility.
> (We have the complete code history, though. Just not
> easily loadable)
> - The changes file would hit the 32MB limite before you would get
> a complete 39beta even if it would work in theory.
>
> Besides keeping the source, I'm sure you would want to keep
> the changeset information, too. Cost: 5MB of image size for 3.9 alone.
>
> So just for a 3.9 with all changes, all in all we are talking about
> 30MB that nobody needs but system developers. Do we *really*
> want to ship that to all the users?
>
> The history mechanism of Squeak does not scale anyway. Why
> only ship the history from 3.0? why not the *complete* history? What
> do you do in 10 years? 7MB left in the changeset is kind of small...
>
> The whole history mechanism of Squeak does not scale: There
> is no need at all to have the old versions on the local disc at all.
> What we need is a server that has the complete history that
> then can be queried from the clients, everthing else makes
> no sense.
>
> For now, we should keep of course the latest beta version with the
> history of
> 3.9 available. Just that it's not called "3.9", but "3.9 dev" or
> something
> like that.
>
> And we should get a student to work on that code history project...
>
> Marcus

Marcus Denker

Re: Condensed Sources vs. Squeak Maintainence

On 24.07.2006, at 11:54, Klaus D. Witzel wrote:

> Hi Marcus,
>
> since every other project (package) depends on the integrity of the
> source code mechanism, I hereby volunteer to get a new source code
> subsystem (incl. code history) done during this summer. We have no
> time to wait for a new (reliable, scalable, available) source code
> mechanism, so I intend to make resources available for this project.
>

Good! But we should not wait with 3.9 for this.

> Here's a draft for a new source code subsystem:
>
> 0] (digit for Wolfgang ;-) there are at least three preferences:
>
> 1] use the traditional subsystem as is
>
> 2] use Dan Ingalls Source Compression as suggested earlier
>
> 3] use a service based (network based) mechanism
>
> I've studied the recent additions which where made to
> CompiledMethod w.r.t. MethodProperties, this has to be cleaned up
> before anything else can be done.
> No, I'm not a friend of storing any MetaData *in* the meta data's
> object.

By the selector of a Method is not "meta"! Every method should have a
name, and if you want a method with another name, make a new method.
Even the very idea of
different names for the same method is not a good one, as the name is
hard-coded in the sourcecode, too. It's a direct, non-meta property
of the method object.

> I propose to put the MethodProperties into the MethodDictionary
> where they belong (same method can *and will* have different
> properties per class

Methods should be only installed in one class. yes... Traits, sigh...
the copying of Traits is evil... I think a better model for traits
would have been to share Bytecode,
not Methods. But then in Squeak these two concepts have been mangled
into one for space reasons, which was a bad idea in general. (Or: It
was a bad idea not
to harvest Tims fix in 1998...).

> = one MethodProperties in each class' method dictionary). As a
> consequence, today's MethodProperties would all be garbage
> collected (except for the few pragmas), because all they store is
> the selector. And CompiledMethod>>#selector is resolved by
> #methodClass and #keyAtValue: so #who can disappear as is intended.
>
> After that, the *new* source code information can be put into the
> new MethodProperties, which allows an easy migration because the
> old source code pointer won't be touched until after migration.
>

You can do that even in the current structure. The sourcePointer is
now per Method, too.

Marcus

smime.p7s (5K) Download Attachment

Edgar J. De Cleene

Re: Condensed Sources vs. Squeak Maintainence

In reply to this post by Klaus D. Witzel

Klaus D. Witzel puso en su mail :

> 3] use a service based (network based) mechanism
+1

And again I ask if we could have a official class repository from where all
could dowload

Edgar

__________________________________________________
Preguntá. Respondé. Descubrí.
Todo lo que querías saber, y lo que ni imaginabas,
está en Yahoo! Respuestas (Beta).
¡Probalo ya!
http://www.yahoo.com.ar/respuestas

Andreas.Raab

Re: Condensed Sources vs. Squeak Maintainence

In reply to this post by Klaus D. Witzel

> Comments, suggestions, critique, all appreciated.

Just one: The idea sounds great but can we keep the entirely unrelated
issues of a new source code subsystem and the particular implementation
of per-method properties separate? I don't see why a new source code
subsystem would require a particular implementation of per-method
properties and having the two issues separate allows us to discuss and
assess the merits independently.

Cheers,
- Andreas

Klaus D. Witzel wrote:

> Hi Marcus,
>
> since every other project (package) depends on the integrity of the
> source code mechanism, I hereby volunteer to get a new source code
> subsystem (incl. code history) done during this summer. We have no time
> to wait for a new (reliable, scalable, available) source code mechanism,
> so I intend to make resources available for this project.
>
> Here's a draft for a new source code subsystem:
>
> 0] (digit for Wolfgang ;-) there are at least three preferences:
>
> 1] use the traditional subsystem as is
>
> 2] use Dan Ingalls Source Compression as suggested earlier
>
> 3] use a service based (network based) mechanism
>
> I've studied the recent additions which where made to CompiledMethod
> w.r.t. MethodProperties, this has to be cleaned up before anything else
> can be done.
> No, I'm not a friend of storing any MetaData *in* the meta data's
> object.
> I propose to put the MethodProperties into the MethodDictionary where
> they belong (same method can *and will* have different properties per
> class = one MethodProperties in each class' method dictionary). As a
> consequence, today's MethodProperties would all be garbage collected
> (except for the few pragmas), because all they store is the selector.
> And CompiledMethod>>#selector is resolved by #methodClass and
> #keyAtValue: so #who can disappear as is intended.
>
> After that, the *new* source code information can be put into the new
> MethodProperties, which allows an easy migration because the old source
> code pointer won't be touched until after migration.
>
> Comments, suggestions, critique, all appreciated.
>
> /Klaus
>
> On Mon, 24 Jul 2006 09:16:58 +0200, Marcus wrote:
>
>>
>> On 24.07.2006, at 08:46, Peace Jerome wrote:
>>
>>> Hi all,
>>>
>>>
>>> I've noticed that condensing sources is on the todo
>>> list of the 3dot9 gamma push.
>>>
>>
>> One of the problems is that the .changes is limited to 32 MB,
>> and 3.9 7048 is already 25MB, which means people would
>> very soon see the limit in normal use (e.g. when loading a bunch
>> of packages).
>>
>> Another problem is that if we remove the linefeeds in those
>> couple of hundrets of methods and do a bit of more _ conversion
>> (which is intended), then we even hit the 32MB limit while
>> in 3.9alpha.
>>
>>> One of the deliverables with 3dot9 final should be a
>>> 3.10 with all changes from 3.0.
>>>
>>
>> This is not possible because of two reasons:
>> - the MC based process is broken, we can't load from 3.0
>> to get 3.9beta. Yes, bad. I am taking full responsibility.
>> (We have the complete code history, though. Just not
>> easily loadable)
>> - The changes file would hit the 32MB limite before you would get
>> a complete 39beta even if it would work in theory.
>>
>> Besides keeping the source, I'm sure you would want to keep
>> the changeset information, too. Cost: 5MB of image size for 3.9 alone.
>>
>> So just for a 3.9 with all changes, all in all we are talking about
>> 30MB that nobody needs but system developers. Do we *really*
>> want to ship that to all the users?
>>
>> The history mechanism of Squeak does not scale anyway. Why
>> only ship the history from 3.0? why not the *complete* history? What
>> do you do in 10 years? 7MB left in the changeset is kind of small...
>>
>> The whole history mechanism of Squeak does not scale: There
>> is no need at all to have the old versions on the local disc at all.
>> What we need is a server that has the complete history that
>> then can be queried from the clients, everthing else makes
>> no sense.
>>
>> For now, we should keep of course the latest beta version with the
>> history of
>> 3.9 available. Just that it's not called "3.9", but "3.9 dev" or
>> something
>> like that.
>>
>> And we should get a student to work on that code history project...
>>
>> Marcus
>
>
>
>

Klaus D. Witzel

Re: Condensed Sources vs. Squeak Maintainence

In reply to this post by Marcus Denker

Hi Marcus,

on Mon, 24 Jul 2006 12:11:31 +0200, you wrote:

> On 24.07.2006, at 11:54, Klaus D. Witzel wrote:
>
>> Hi Marcus,
>>
>> since every other project (package) depends on the integrity of the
>> source code mechanism, I hereby volunteer to get a new source code
>> subsystem (incl. code history) done during this summer. We have no
>> time to wait for a new (reliable, scalable, available) source code
>> mechanism, so I intend to make resources available for this project.
>>
>
> Good! But we should not wait with 3.9 for this.

O.K.

>> Here's a draft for a new source code subsystem:
>>
>> 0] (digit for Wolfgang ;-) there are at least three preferences:
>>
>> 1] use the traditional subsystem as is
>>
>> 2] use Dan Ingalls Source Compression as suggested earlier
>>
>> 3] use a service based (network based) mechanism
>>
>> I've studied the recent additions which where made to
>> CompiledMethod w.r.t. MethodProperties, this has to be cleaned up
>> before anything else can be done.
>> No, I'm not a friend of storing any MetaData *in* the meta data's
>> object.
>
> By the selector of a Method is not "meta"!

I'm sorry it is "meta" by tradition. Otherwise it would have been stored
in the method (literals) since the 80's of the previous century. And if
you cannot agree with that view:

only the VM knows about the relationship between the selector used for
method lookup and the found and executed method. This *is* what meta is
about, that's what I think.

> Every method should have a
> name, and if you want a method with another name, make a new method.

No clash here: if I throw the MethodProperties oop out of CompiledMethod's
literals then I can still fulfill this your requirement (and you owe me a
beer ;-).

> Even the very idea of
> different names for the same method is not a good one, as the name is
> hard-coded in the sourcecode, too. It's a direct, non-meta property
> of the method object.

Sorry (same argument as above). It is, and I have to agree, a
*substantial* part of the source-to-compiled method messenger "RNA"
enzymatic process. But the result cannot tell which name its gene had for
it (we human beings can, same for source/compiled code, to some extent).
Take as a counter example #DoItwith:with:... and the cause for having
CompiledMethod>>defaultSelector.

But anyways, I do not see a clash here and it's only a matter
implementation.

Having said that, was is really missing is " an
ObsoleteCompiledMethod(4711) ", mantained by MethodDictionary with key :=
oldMethod and value := aClass -> aSelector. A new global variable
ObsoleteCompiledMethods can hold, with advantage, an instance of
WeakIdentityKeyDictionary for these keys and values.

>> I propose to put the MethodProperties into the MethodDictionary
>> where they belong (same method can *and will* have different
>> properties per class
>
> Methods should be only installed in one class.

And the system should be transparent if not so. But again, it can be
implemented the way you describe.

> yes... Traits, sigh...
> the copying of Traits is evil... I think a better model for traits
> would have been to share Bytecode,
> not Methods. But then in Squeak these two concepts have been mangled
> into one for space reasons, which was a bad idea in general. (Or: It
> was a bad idea not
> to harvest Tims fix in 1998...).

I have no problem with this. This will (perhaps) be reviewed once people
take advantage of Squeak Trait(s).

>> = one MethodProperties in each class' method dictionary). As a
>> consequence, today's MethodProperties would all be garbage
>> collected (except for the few pragmas), because all they store is
>> the selector. And CompiledMethod>>#selector is resolved by
>> #methodClass and #keyAtValue: so #who can disappear as is intended.
>>
>> After that, the *new* source code information can be put into the
>> new MethodProperties, which allows an easy migration because the
>> old source code pointer won't be touched until after migration.
>>
>
> You can do that even in the current structure. The sourcePointer is
> now per Method, too.

? I was referring to the old (as is now) and, at the same time, during
migration, to the then new sourcePointer, with emphasis on having both
available during migration. What was it what you meant?

/Klaus

>
> Marcus

Klaus D. Witzel

Re: Condensed Sources vs. Squeak Maintainence

In reply to this post by Andreas.Raab

On Mon, 24 Jul 2006 14:00:16 +0200, Andreas Raab wrote:

> > Comments, suggestions, critique, all appreciated.
>
> Just one: The idea sounds great but can we keep the entirely unrelated
> issues of a new source code subsystem and the particular implementation
> of per-method properties separate? I don't see why a new source code
> subsystem would require a particular implementation of per-method
> properties and having the two issues separate allows us to discuss and
> assess the merits independently.

You're right, and thank you for pointing me to the independence of the two
issues. As everybody else in software industry, I was suggesting a
particular implementation (of new sourcePointer) instead of a sound and
complete design ;-)

/Klaus

> Cheers,
> - Andreas

Marcus Denker

Re: Condensed Sources vs. Squeak Maintainence

In reply to this post by Klaus D. Witzel

On 24.07.2006, at 18:21, Klaus D. Witzel wrote:

>>
>> By the selector of a Method is not "meta"!
>
> I'm sorry it is "meta" by tradition. Otherwise it would have been
> stored in the method (literals) since the 80's of the previous
> century. And if you cannot agree with that view:
>
> only the VM knows about the relationship between the selector used
> for method lookup and the found and executed method. This *is* what
> meta is about, that's what I think.
>

If the model that we want to have is just "execution", then this is
correct. This is what we have now: A model describing classes/
methods, where the description of the
classes is both for the machine and the programmer, but for methods
we only model execution. What I would like to see (now that machines
have become so
much more powerful) is to move this model up one level and make it's
main goal to be a model of the program for engineering, while keeping
this model executionable.

This is why a method should know it's class (this is what a developer
thinks about a method) and it's name... the whole mode of the system
should be on an abstraction
level of that kind of model that you would design for a re-
engineering or visualisation tool...

e.g. for execution, a class needs no name, just like a method needs
no name. You could have nameless "CompiledClasses". But the objects
encoding classes we do
have a slot for the name... Because they have not only the
interpreter as the client, but the developer tools (and the
developer), too. Because it makes no sense to
destinguish between the two: The higher level model with name is
perfectly executable, too.

It makes the tools much simpler: In 3.9, methods know their class
without having to search the whole system. As soon as that was done,
I started to refactor a little... it
turns out that this information makes the design simpler: e.g no need
to call "decompileClass: aClass selector: selector" on the
compiledMethod... just call
decompile on the method, no need to funnel the extra information to
all the places where they are needed. For the tools, it feels much
more natural if the methods
know more than nothing... as then the class/method structure will be
your meta model that you can use directly.

Marcus

smime.p7s (5K) Download Attachment

Klaus D. Witzel

Re: Condensed Sources vs. Squeak Maintainence

On Mon, 24 Jul 2006 19:44:01 +0200, Marcus Denker wrote:

> On 24.07.2006, at 18:21, Klaus D. Witzel wrote:
>
>
>>>
>>> By the selector of a Method is not "meta"!
>>
>> I'm sorry it is "meta" by tradition. Otherwise it would have been
>> stored in the method (literals) since the 80's of the previous
>> century. And if you cannot agree with that view:
>>
>> only the VM knows about the relationship between the selector used
>> for method lookup and the found and executed method. This *is* what
>> meta is about, that's what I think.
>>
>
> If the model that we want to have is just "execution", then this is
> correct. This is what we have now: A model describing classes/
> methods, where the description of the
> classes is both for the machine and the programmer, but for methods
> we only model execution. What I would like to see (now that machines
> have become so
> much more powerful) is to move this model up one level and make it's
> main goal to be a model of the program for engineering, while keeping
> this model executionable.

I'm in sync.

> This is why a method should know it's class (this is what a developer
> thinks about a method) and it's name... the whole mode of the system
> should be on an abstraction
> level of that kind of model that you would design for a re-
> engineering or visualisation tool...

But by using methodClass (to which I fully agree), every method knows it
class, which knows its method dictionary, which knows the corresponding
selector (all a matter of implementation). No need for MethodProperties
here.

And yes (and to make it clear): every method nowadays has methodClass
(except perhaps those circumscribed in a comment, similiar to 'bad case'
in the other thread). Since any two ~= classes cannot be the same type, so
their methods. So methodClass *must* be checked for by CompiledMethod>>#=
... (which, BTW, doesn't check the selector, in any version I've seen so
far ;-)

> e.g. for execution, a class needs no name, just like a method needs
> no name. You could have nameless "CompiledClasses". But the objects
> encoding classes we do
> have a slot for the name...

Good point. This is equivalent to (Smalltalk keyAtValue: aSlot), same as
for (aMethodDictionary keyAtValue: aCompiledMethod), a matter of
performance at the time of invention (CompiledMethod instanceCount > (...
allObjectsdo: ... isBehavior ...) size, a subject for lengthy discussions
in modern times ;-)

> Because they have not only the
> interpreter as the client, but the developer tools (and the
> developer), too. Because it makes no sense to
> destinguish between the two: The higher level model with name is
> perfectly executable, too.

Sure, the higher level is "just" about *organization*, *reuse* and other
elaborate concepts, a typical reason for introducing execution unrelated
meta data (similiar to ClassDescription).

> It makes the tools much simpler: In 3.9, methods know their class
> without having to search the whole system. As soon as that was done,
> I started to refactor a little... it
> turns out that this information makes the design simpler: e.g no need
> to call "decompileClass: aClass selector: selector" on the
> compiledMethod... just call
> decompile on the method, no need to funnel the extra information to
> all the places where they are needed.

Yes, when I saw that it was quite a surprise how easy things can be ;-)
Good reason for marking these oldies with #deprecated:.

> For the tools, it feels much
> more natural if the methods
> know more than nothing... as then the class/method structure will be
> your meta model that you can use directly.

O.K. but the level we are talking about here is equivalent to [at least]
Behavior>>#allCallsOn: why the heck should Behavior know anything about
calls to symbols in CompiledMethods? It delegates.

So can CompiledMethod (delegate) when asked the queries you mentioned. It
doesn't need to store anything else than Blue Book's methodClass for
obtaining these answers (by delegation).

I second implementing all possible queries and oops related to
CompiledMethod in CompiledMethod. Perhaps as a separator for a redesign:
oops which are unrelated to CompiledMethod>>#= (unrelated to type-,
function-, computational-equivalency "PCE") do not belong to
CompiledMethod. A few examples for the unrelated: #categoryOfElement:,
#tempNames, source-origin-server, source-origin-path, source-origin-uri,
source-origin-version, ..., source-origin-name (selector) they shouted
when the baby was born ;-)

/Klaus

> Marcus

stéphane ducasse-2

Re: Condensed Sources vs. Squeak Maintainence

In reply to this post by Marcus Denker

>
>> One of the deliverables with 3dot9 final should be a
>> 3.10 with all changes from 3.0.
>>
>
> This is not possible because of two reasons:
> - the MC based process is broken, we can't load from 3.0
> to get 3.9beta. Yes, bad. I am taking full responsibility.

I'm too.
I invite everybody to build something and we will use it.

> (We have the complete code history, though. Just not
> easily loadable)
> - The changes file would hit the 32MB limite before you would get
> a complete 39beta even if it would work in theory.
>
> Besides keeping the source, I'm sure you would want to keep
> the changeset information, too. Cost: 5MB of image size for 3.9 alone.
>
> So just for a 3.9 with all changes, all in all we are talking about
> 30MB that nobody needs but system developers. Do we *really*
> want to ship that to all the users?
>
> The history mechanism of Squeak does not scale anyway. Why
> only ship the history from 3.0? why not the *complete* history? What
> do you do in 10 years? 7MB left in the changeset is kind of small...
>
> The whole history mechanism of Squeak does not scale: There
> is no need at all to have the old versions on the local disc at all.
> What we need is a server that has the complete history that
> then can be queried from the clients, everthing else makes
> no sense.
>
> For now, we should keep of course the latest beta version with the
> history of
> 3.9 available. Just that it's not called "3.9", but "3.9 dev" or
> something
> like that.
>
> And we should get a student to work on that code history project...
>
> Marcus

stéphane ducasse-2

Re: Condensed Sources vs. Squeak Maintainence

In reply to this post by Andreas.Raab

+ 1
Changes are necessary but costly so we should go stepwise.

Stef

> > Comments, suggestions, critique, all appreciated.
>
> Just one: The idea sounds great but can we keep the entirely
> unrelated issues of a new source code subsystem and the particular
> implementation of per-method properties separate? I don't see why a
> new source code subsystem would require a particular implementation
> of per-method properties and having the two issues separate allows
> us to discuss and assess the merits independently.
>
> Cheers,
> - Andreas
>
>
> Klaus D. Witzel wrote:
>> Hi Marcus,
>> since every other project (package) depends on the integrity of
>> the source code mechanism, I hereby volunteer to get a new source
>> code subsystem (incl. code history) done during this summer. We
>> have no time to wait for a new (reliable, scalable, available)
>> source code mechanism, so I intend to make resources available for
>> this project.
>> Here's a draft for a new source code subsystem:
>> 0] (digit for Wolfgang ;-) there are at least three preferences:
>> 1] use the traditional subsystem as is
>> 2] use Dan Ingalls Source Compression as suggested earlier
>> 3] use a service based (network based) mechanism
>> I've studied the recent additions which where made to
>> CompiledMethod w.r.t. MethodProperties, this has to be cleaned up
>> before anything else can be done.
>> No, I'm not a friend of storing any MetaData *in* the meta
>> data's object.
>> I propose to put the MethodProperties into the MethodDictionary
>> where they belong (same method can *and will* have different
>> properties per class = one MethodProperties in each class' method
>> dictionary). As a consequence, today's MethodProperties would all
>> be garbage collected (except for the few pragmas), because all
>> they store is the selector. And CompiledMethod>>#selector is
>> resolved by #methodClass and #keyAtValue: so #who can disappear as
>> is intended.
>> After that, the *new* source code information can be put into the
>> new MethodProperties, which allows an easy migration because the
>> old source code pointer won't be touched until after migration.
>> Comments, suggestions, critique, all appreciated.
>> /Klaus
>> On Mon, 24 Jul 2006 09:16:58 +0200, Marcus wrote:
>>>
>>> On 24.07.2006, at 08:46, Peace Jerome wrote:
>>>
>>>> Hi all,
>>>>
>>>>
>>>> I've noticed that condensing sources is on the todo
>>>> list of the 3dot9 gamma push.
>>>>
>>>
>>> One of the problems is that the .changes is limited to 32 MB,
>>> and 3.9 7048 is already 25MB, which means people would
>>> very soon see the limit in normal use (e.g. when loading a bunch
>>> of packages).
>>>
>>> Another problem is that if we remove the linefeeds in those
>>> couple of hundrets of methods and do a bit of more _ conversion
>>> (which is intended), then we even hit the 32MB limit while
>>> in 3.9alpha.
>>>
>>>> One of the deliverables with 3dot9 final should be a
>>>> 3.10 with all changes from 3.0.
>>>>
>>>
>>> This is not possible because of two reasons:
>>> - the MC based process is broken, we can't load from 3.0
>>> to get 3.9beta. Yes, bad. I am taking full responsibility.
>>> (We have the complete code history, though. Just not
>>> easily loadable)
>>> - The changes file would hit the 32MB limite before you
>>> would get
>>> a complete 39beta even if it would work in theory.
>>>
>>> Besides keeping the source, I'm sure you would want to keep
>>> the changeset information, too. Cost: 5MB of image size for 3.9
>>> alone.
>>>
>>> So just for a 3.9 with all changes, all in all we are talking about
>>> 30MB that nobody needs but system developers. Do we *really*
>>> want to ship that to all the users?
>>>
>>> The history mechanism of Squeak does not scale anyway. Why
>>> only ship the history from 3.0? why not the *complete* history? What
>>> do you do in 10 years? 7MB left in the changeset is kind of small...
>>>
>>> The whole history mechanism of Squeak does not scale: There
>>> is no need at all to have the old versions on the local disc at all.
>>> What we need is a server that has the complete history that
>>> then can be queried from the clients, everthing else makes
>>> no sense.
>>>
>>> For now, we should keep of course the latest beta version with the
>>> history of
>>> 3.9 available. Just that it's not called "3.9", but "3.9 dev" or
>>> something
>>> like that.
>>>
>>> And we should get a student to work on that code history project...
>>>
>>> Marcus
>
>

Andreas.Raab

Re: Condensed Sources vs. Squeak Maintainence

In reply to this post by Klaus D. Witzel

Klaus D. Witzel wrote:
> So can CompiledMethod (delegate) when asked the queries you mentioned.
> It doesn't need to store anything else than Blue Book's methodClass for
> obtaining these answers (by delegation).

Not quite true. It is easy to see that in a situation like here:

Object compile:'foo [42]'.
fooBlock := Object new foo.
Object compile:'foo ^21'.

you can't (without major modifications of classes) determine the
properties of "fooBlock home" (say, for source retrieval). This is a
relevant pattern as I've found in the past when dealing with changes to
compiled methods - while not exactly common it is also not rare enough
that it can be easily ignored.

Cheers,
- Andreas

Klaus D. Witzel

Still about methodClass [was: Condensed Sources vs. Squeak Maintainence]

On Tue, 25 Jul 2006 08:53:49 +0200, Andreas Raab wrote:

> Klaus D. Witzel wrote:
>> So can CompiledMethod (delegate) when asked the queries you mentioned.
>> It doesn't need to store anything else than Blue Book's methodClass for
>> obtaining these answers (by delegation).
>
> Not quite true. It is easy to see that in a situation like here:
>
> Object compile:'foo [42]'.
> fooBlock := Object new foo.
> Object compile:'foo ^21'.
>
> you can't (without major modifications of classes) determine the
> properties of "fooBlock home" (say, for source retrieval).

This one (and recompiles while in debugger, fileIn's, another browser,
parallel to a forked process, etc) can easily be addressed by the new
ObsoleteCompiledMethods dictionary which I mentioned earlier. No major
change (no change at all :) to classes necessary, just a proper action
during aMethodDictionary xyzAt:ifAbsent:.

I could give more [boring] details if that's wanted.

> This is a relevant pattern as I've found in the past when dealing with
> changes to compiled methods - while not exactly common it is also not
> rare enough that it can be easily ignored.

Yes, neglection of these would have to be categorized under
'major-failure'.

/Klaus

> Cheers,
> - Andreas
>
>
>
>

Andreas.Raab

Re: Still about methodClass [was: Condensed Sources vs. Squeak Maintainence]

Klaus D. Witzel wrote:
> I could give more [boring] details if that's wanted.

Actually, if you want my attention, talk about the new source code
subsystem ;-)

The method properties discussion is a red herring in my eyes - we have a
functioning version today and if that needs changing or not is in the
eye of the beholder. Personally (having done it both ways) I strongly
prefer the current solution since it has some nice properties (like that
all relevant attributes are reachable via direct references from the
method and therefore garbage collection will do The Right Thing without
additional help; that a generic copying mechanism doesn't have to know
that some vital parts of compiled methods are stored elsewhere; that
extensions can be done in one place rather than many) but things can
certainly be done differently. Discussing those merits would require
some more data points other than saying "yes, it can be done" (because I
know it can, I have done it before ;-) and would likely mean some fairly
subjective comparisons of what is "nicer" or "more understandable" or
somesuch.

In light of that, I'd rather discuss an area that actually needs fixing,
like the source code subsystem.

Cheers,
- Andreas

Klaus D. Witzel

Re: Still about methodClass

On Tue, 25 Jul 2006 17:19:11 +0200, Andreas Raab wrote:

> Klaus D. Witzel wrote:
>> I could give more [boring] details if that's wanted.
>
> Actually, if you want my attention, talk about the new source code
> subsystem ;-)

So we'll take branch here, the new source code subsystem topic will get a
new thread.

> The method properties discussion is a red herring in my eyes - we have a
> functioning version today and if that needs changing or not is in the
> eye of the beholder. Personally (having done it both ways) I strongly
> prefer the current solution since it has some nice properties (like that
> all relevant attributes are reachable via direct references from the
> method and therefore garbage collection will do The Right Thing without
> additional help; that a generic copying mechanism doesn't have to know
> that some vital parts of compiled methods are stored elsewhere; that
> extensions can be done in one place rather than many) but things can
> certainly be done differently. Discussing those merits would require
> some more data points other than saying "yes, it can be done" (because I
> know it can, I have done it before ;-) and would likely mean some fairly
> subjective comparisons of what is "nicer" or "more understandable" or
> somesuch.

Good points. May I stress that my concern is the *current* *integrity* of

(nil systemNavigation
allMethodsSelect: [:method |
method hasNewPropertyFormat == false]) size
=> 0

|x y | x := y := 0.
nil systemNavigation
allMethodsSelect: [:method |
method literals last isVariableBinding
ifTrue: [x := 1 + x]
ifFalse: [y := 1 + y]. false].
x @ y => 49487@0

|x y | x := y := 0.
CompiledMethod allInstancesDo: [:z |
z literals last isVariableBinding
ifTrue: [x := 1 + x]
ifFalse: [y := 1 + y]].
x@y => 49035@5

(note the discrepancy, 49487 versus 49035, 0 versus 5, relative to
3.9#7048)

As discussed with Marcus last night (1st Squeak Stammtisch in Bern, 4
Squeakers :) I'm no longer in opposition to how and where MethodProperties
are stored and maintained. And if I understand your comment above
correctly, that item is hereby closed.

But aCompiledMethod methodClass (aMethod literals last isVariableBinding)
must be solved (and perhaps the related implementation must be reviewed)
before I'm interested in going a step further into the direction of a new
source code subsystem. My intention here is that there is consensus on WHO
exactly tells which class a method belongs to; when can (if at all) that
change; how are obsolete methods managed in the future. This must be as
reliable as possible, otherwise we can forget about aCompiledMethod>>#=
for use in any other (sub)system. No, it's not my intention to invent
something which is ersatz (<= WordNet/2.1) for aCompiledMethod>>#=.

A nice and bad (perhaps no *that* close but nevertheless frightening)
example is ChangeRecord>>#methodClass, it asks Smalltalk (regardless of
obsolence)...

In other words and in a single statement: clean up the #methodClass mess
in the image.

Thereafter we can introduce (and/or reuse) concepts which support a source
code view on current, obsolete, historized and future CompiledMethods, a'
la distributed authoring and versioning.

Please nobody misunderstand me here: propose a fix for #methodClass
(obsolete methods as well) and we can immediately turn over to the more
interesting things like managing the future (so to speak).

> In light of that, I'd rather discuss an area that actually needs fixing,
> like the source code subsystem.

Me too; let's fix first things first.

/Klaus

> Cheers,
> - Andreas
>
>

Marcus Denker

Re: Still about methodClass

On 26.07.2006, at 07:36, Klaus D. Witzel wrote:

> On Tue, 25 Jul 2006 17:19:11 +0200, Andreas Raab wrote:
>
>> Klaus D. Witzel wrote:
>>> I could give more [boring] details if that's wanted.
>>
>> Actually, if you want my attention, talk about the new source code
>> subsystem ;-)
>
> So we'll take branch here, the new source code subsystem topic will
> get a new thread.
>
>> The method properties discussion is a red herring in my eyes - we
>> have a functioning version today and if that needs changing or not
>> is in the eye of the beholder. Personally (having done it both
>> ways) I strongly prefer the current solution since it has some
>> nice properties (like that all relevant attributes are reachable
>> via direct references from the method and therefore garbage
>> collection will do The Right Thing without additional help; that a
>> generic copying mechanism doesn't have to know that some vital
>> parts of compiled methods are stored elsewhere; that extensions
>> can be done in one place rather than many) but things can
>> certainly be done differently. Discussing those merits would
>> require some more data points other than saying "yes, it can be
>> done" (because I know it can, I have done it before ;-) and would
>> likely mean some fairly subjective comparisons of what is "nicer"
>> or "more understandable" or somesuch.
>
> Good points. May I stress that my concern is the *current*
> *integrity* of
>
> (nil systemNavigation
> allMethodsSelect: [:method |
> method hasNewPropertyFormat == false]) size
> => 0

All methods have been recompiled to use the new format (with class
binding in last literal)

>
> |x y | x := y := 0.
> nil systemNavigation
> allMethodsSelect: [:method |
> method literals last isVariableBinding
> ifTrue: [x := 1 + x]
> ifFalse: [y := 1 + y]. false].
> x @ y => 49487@0
>
> |x y | x := y := 0.
> CompiledMethod allInstancesDo: [:z |
> z literals last isVariableBinding
> ifTrue: [x := 1 + x]
> ifFalse: [y := 1 + y]].
> x@y => 49035@5
>
> (note the discrepancy, 49487 versus 49035,

non-locally intalled traits methods.

> 0 versus 5, relative to 3.9#7048)
>

There are some methods on the stack that have not yet finished
executing... should be fixed.
It's on the never-ending list of stuff on the todo.

> As discussed with Marcus last night (1st Squeak Stammtisch in Bern,
> 4 Squeakers :) I'm no longer in opposition to how and where
> MethodProperties are stored and maintained. And if I understand
> your comment above correctly, that item is hereby closed.
>
> But aCompiledMethod methodClass (aMethod literals last
> isVariableBinding) must be solved (and perhaps the related
> implementation must be reviewed) before I'm interested in going a
> step further into the direction of a new source code subsystem. My
> intention here is that there is consensus on WHO exactly tells
> which class a method belongs to; when can (if at all) that change;
> how are obsolete methods managed in the future. This must be as
> reliable as possible, otherwise we can forget about
> aCompiledMethod>>#= for use in any other (sub)system. No, it's not
> my intention to invent something which is ersatz (<= WordNet/2.1)
> for aCompiledMethod>>#=.
>

#= does not take the Class of the method into account, as any client
for equality would not want that. You want to ask "which methods are
the same in the system". If we
take the Class into account, you will never find any equeal methods...

> A nice and bad (perhaps no *that* close but nevertheless
> frightening) example is ChangeRecord>>#methodClass, it asks
> Smalltalk (regardless of obsolence)...
> words and in a single statement: clean up the #methodClass mess in
> the image.
>

The whole fileOut / Changes stuff os a huge huge mess. But
#methodClass in ChangeRecord has not much to do with #methodClass on
CompiledMethod...

Seriously, the only way to improve the situation is to improve it...
we had many people telling that only perfect solutions are
acceptable. Or worthy of Squeak
or something. This is wrong: We need *better* solutions, not perfect
ones. If we would have just made everything continously a little bit
better over the last
10 years, we would have quite a cool system now.

The idea of searching for the "perfect" solution only (while letting
everything rot) is the reason for a lot of what we see now.

Marcus

smime.p7s (5K) Download Attachment