Squeak Maintainence and Condensed Sources

Squeak Maintainence and Condensed Sources

Hi Marcus,

Thank you for your first reply to
Condensed Sources vs. Squeak Maintainence

To restate the essesnce of my concern:

I believe a great deal of code will rot and squeak
will become a lot more fragile than it already is if
you compress sources now in the process of finalizing
3.9.

I understand your concern about the changes file going
over its limit.

I also understand the great deal of effort you and
stef have put it to 3.9 to date. And that getting
things done has largely relied on your time and
effort.  Easy to implement solutions would be
appreciated I imagine?

First the task of getting a 3.10 with all changes from
3.9 would start with '3.9a with all changes from 3.0'
that Doug Way (bless his heart) produced after I said
please three times in one post.

Second condensed sources should come at the beginning
of an alpha cycle not at an end.  So one alternative
is to condense sources with the first version of 3.9
alpha (or the 3.8 version just before that name
change.  And produce a 3.9 with all changes from 3.8
(condensed) source. That would not be as nice but it
would at least be acceptable.

The full changes versions have never been for users
they have been for developers. Their usefulness is
very high or I wouldn’t be trying so hard to make my
point.

A code history project is both far in the future and
risky (it might not come off).  Do you really want
Squeak maintainence to limp along until when and if it
comes about?

The easier you make it for me (and others) to find the
bugs the quicker, better, and more permanently we can
fix them.

You and Stef have been given arbitrary authority to
determine how 3.9 gets made.  The consequences of how
you choose to use that authority and what decisions
you make affects the whole community.

I see several of those decisions as increasing the
brittleness of squeak and I am voicing my concerns. I
ask only that you do your best to address them.

Yours in service, -- Jerome Peace






__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around
http://mail.yahoo.com 

On 25.07.2006, at 06:15, Peace Jerome wrote:
It is not a "concern". The changefile is 25MB in 7048, this
is not a "concern", this is a fact. And the other fact is that
the maximum size of the changes file is 32MB.


>
> First the task of getting a 3.10 with all changes from
> 3.9 would start with '3.9a with all changes from 3.0'
> that Doug Way (bless his heart) produced after I said
> please three times in one post.
>

This is technically not possible, as the changes file would
go over it's maximum size.

> Second condensed sources should come at the beginning
> of an alpha cycle not at an end.  So one alternative
> is to condense sources with the first version of 3.9
> alpha (or the 3.8 version just before that name
> change.  And produce a 3.9 with all changes from 3.8
> (condensed) source. That would not be as nice but it
> would at least be acceptable.
>

3.9 is not reproducable from 3.8, so building it on top
of some modified 3.8 is impossible, at least if we plan
to ship it soon at all.

shiping 3.9 with a .changes file of >25MB is impossible, too.

So what do you suggest?


     Marcus

On 25 juil. 06, at 06:15, Peace Jerome wrote:

Why do you say that kind of thing?
I really do not get it. This is not true.


But this will not scale!

> Second condensed sources should come at the beginning
> of an alpha cycle not at an end.

So you will not get a true condense version but one with all the changes
from that version.

> So one alternative
> is to condense sources with the first version of 3.9
> alpha (or the 3.8 version just before that name
> change.  And produce a 3.9 with all changes from 3.8
> (condensed) source. That would not be as nice but it
> would at least be acceptable.

> You and Stef have been given arbitrary authority to
> determine how 3.9 gets made.  The consequences of how
> you choose to use that authority and what decisions
> you make affects the whole community.


But are you serious? Do you want us to deliver a system that
you CANNOT use? Because you cannot even load any decent
package? Is it really what you want?

> I see several of those decisions as increasing the
> brittleness of squeak and I am voicing my concerns. I
> ask only that you do your best to address them.

Have you checked the number of bug fixes that we integrated/took care?
Instead of complaining like that the only way to go is:
        - build a simple infrastructure so that we can browse any code version
        - fix the limit of the 32Mb

There are no other choice

Stef

On 25.07.2006, at 08:46, Marcus Denker wrote:

>
>>
>> I believe a great deal of code will rot and squeak
>> will become a lot more fragile than it already is if
>> you compress sources now in the process of finalizing
>> 3.9.


To make one thing clear: I do understand that code history
is *extremely* important. And I do know that we need
even more than a simple per-method history can do.

At SCG, we actually do research into analyzing history
of code... so be assured that we understand why it's needed,
that it is needed, and that without a past there will be no future.

But: Right know the code-history of Squeak is not scalable enough
to provide a history for the whole system, and we need to
balance between what is possible easily now and what is needed
for the future.

     Marcus

I would suggest having two versions, the main one having condensed sources.
Or how about making 3.9a into 3.10 condense the changes and then start 3.11
right away.

It is very important that we have either a condensed change version, or a
condense changes method that works.  The limit is already way to close for
any real work.

Ron Teitelbaum

>
> So what do you suggest?
>
>
>      Marcus
>

Hi Marcus,

Thanks for your replys.

I had carefully crafted a reply to your first
response.
Which your most recent reply may have invalidated.
Still it sums up my best thinking so far on this and
it answers the above question to here it is:

Squeak Maintainence and Condensed Sources

Hi Marcus,

Thank you again for your response. I am feeling
reassured by what you say about the importance of code
history. And by the fact that YOU are saying it.

The ideal future is far away and uncertain.  So I hope
you will give some consideration to facilitating what
we can produce easily with the tools on hand.

Please give consideration to the formula:
-Create a condensed source version for Squeak-6665
(either 3.8 final or 3.9a start).
-Provide the load scripts to produce 3.9final with all
changes from that source.
-And have that as a deliveralble for the squeak
maintenence folks.

It will help.

<Current note: So why is it impossible to do this? --
Jer>

Also it would probalbly be good to draw others into
the pieces of this project you don’t strictly have to
do yourself.  I suspect there are a good number of
squeakers who have enough expericence to condense
Squeak-6665. I don’t know how many can do what you and
stef do with the load scripts. This might be a good
excuse to think about how to transfer that knowledge.

I believe what I have suggested is possible. We need
to take care of what is needed for right now within
the present limitation. The question of what to do
about scalability can be postponed ‘til the future.

Yours in service, --Jerome Peace



>Marcus Denker denker at iam.unibe.ch
>Tue Jul 25 08:29:05 UTC 2006 replied:

>To make one thing clear: I do understand that code
history
>is *extremely* important. And I do know that we need
>even more than a simple per-method history can do.
>
>At SCG, we actually do research into analyzing
history
>of code... so be assured that we understand why it's
needed,
>that it is needed, and that without a past there will
be no future.

>     Marcus

I expect you merely to do your best.
>

--- Marcus Denker <[hidden email]> wrote:

1) I am talking either 3.8 or 3.9a at 6665 point to
present. Is there some practical reason why 6665 would
not be the same beast if the changes were compressed.
Because in theory it should be the same thing with a
different changes and source. Or is it that the MC
load scripts can't reproduce the process of change?

2) I want the all changes version for its value in
finding the RIGHT fixes to future bug discoveries.
This is very valuable. And very important. It is not
as urgent as the finalization process. Which means if
you can find a way to do it. Or if you can find away
to even approximate it. It should be done. It doesn't
have to be done immediately. It has to be done
eventually. Preferably sooner than later. But even
later is valuable.

3) I am speaking up about it now because this is the
time to think of the plan to see it happen. Before the
 bridges are burnt.

 
>
> shiping 3.9 with a .changes file of >25MB is
> impossible, too.

Sigh. I'm a programmer, I am used to frustration. That
usually happens just before a solution appears. I am a
good programmer. I'm not used to defeat. There must be
a way to handle both concerns. Ron Teitelbaum has made
a suggestion. Does it have any merit?



>
> So what do you suggest?
>
>
>      Marcus
>
As my first reply ended:

I expect you merely to do your best.


Yours in service, --Jerome Peace

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around
http://mail.yahoo.com 

if
>> you compress sources now in the process of
finalizing
>> 3.9.
>
>Why do you say that kind of thing?
>I really do not get it.
>This is not true.

It is true that it is my current belief.

In building Squeak you have included several major
integrations.
-Traits,
-UIManager (and tools and plustools systems.)
-The Squeakland progress from 3.8 including pieces of
stuff from connectors and Kadema
-The Smalland stuff from Diego.
-My iteration on polygons and curves.
-Omnibrowser.

And you have commited to a new way to maintain the
source code MC packages.

I can’t speak for traits because I have not focused on
that aspect of the system.

Omnibrowser from a preliminary look at the code looks
like it has been written by people who know good code.
And though I’ve found some integration bugs they seem
to have been easily fixed.

The UIManager was intergrated with several easily
findable annoying bugs still present.  For a while it
broke the sender/implementers buttons in browsers.
For these bugs to slip by the maintainers would
suggest it was poorly check before being submitted for
integration. Those bugs were fixed.  There are
probably many more subtle bugs attributable to those
packages waiting to be found.

The Squeakland stuff is a mismash. The code there has
been patched and fixed by people still learning the
subtleties of squeak.  Bugs have turned up and more
are likely to be in that code.

The history was important to me in tracking down why
polygons were getting deleted when dropped on the
wrong objects. It helped pinpoint what changed and who
changed it. It helped me get in touch with the
changer. And he used the feedback to improve his code.
With out the history will finding the source of the
problems be as easy?

The Smalland stuff added a lot to squeak in its
usability and visual design.  If you look at the
implementation you see a lack of consideration for
subtleties.  In particular code dependencies were
increased greatly.  That stuff needs to be looked at
closely by someone aware of what it means to reduce
code surface area.  Is that going to be easier to do
if the history is lost?

>
>> I understand your concern about the changes file
going
>> over its limit.
>>
>> I also understand the great deal of effort you and
>> Stef have put it to 3.9 to date. And that getting
>> things done has largely relied on your time and
>> effort.  Easy to implement solutions would be
>> appreciated I imagine?
>>
>> First the task of getting a 3.10 with all changes
from
>> 3.9 would start with '3.9a with all changes from
3.0'
>> that Doug Way (bless his heart) produced after I
said
>> please three times in one post.
>
>But this will not scale!

If it doesn’t scale then you have to stop at
somepoint. The end of a development cycle is the worst
place to stop.  The beginning of one is better.

>
>> Second condensed sources should come at the
beginning
>> of an alpha cycle not at an end.
>
>So you will not get a true condense version but one
with all the changes
>from that version.

Yes.  You would have a 3.9 with all changes from
Squeak-6665. This gives a way to check on the history
of changes in 3.9. That’s what I need. My experience
with Doug’ 6665 with all changes is that it is a BIG
help.  It meant being able to track down programers
intentions and determining where and when things went
wrong.

This does not preclude a second condensing of sources
at the beginning of 3.10. It solves in a less than
ideal but acceptable way the timely need for the
historical data.

>
>> So one alternative
>> is to condense sources with the first version of
3.9
>> alpha (or the 3.8 version just before that name
>> change.  And produce a 3.9 with all changes from
3.8
>> (condensed) source. That would not be as nice but
it
>> would at least be acceptable.
>
>> You and Stef have been given arbitrary authority to
>> determine how 3.9 gets made.  The consequences of
how
>> you choose to use that authority and what decisions
>> you make affects the whole community.
>
>
>But are you serious? Do you want us to deliver a
system that
>you CANNOT use?

I think we are having language problems again. How do
I make myself clear.
What I am asking for is for the sake of maintainence.

3.8 was delivered.  Then the version with the full
changes was made secondarily. It was made after 3-6
months delay when Doug responded to my second request.
I am just asking that my need for this change history
be considered. I am hoping it can be honored at an
earlier point in time to increase its usefulness.

I expect 3.9 to be delivered. And a 3.9 with full
changes to be made available at the same time or
shortly there after.


> Because you cannot even load any decent
>package?

Huh? When do you run out of the ability to load decent
packages?

>Is it really what you want?

If I do not seem to you to be making a reasonable
request, then I must be saying something wrong.
I appologize for the language barrier. (See my reply
to Marcus)
>
>> I see several of those decisions as increasing the
>> brittleness of squeak and I am voicing my concerns.
I
>> ask only that you do your best to address them.
>
>Have you checked the number of bug fixes that we
integrated/took care?

Yes.  You and Marcus have been comprehensive in you
inclusion of fixes and enhancements into 3dot9.  You
deserve the credit for the good work and effort.

>Instead of complaining like that the only way to go
is:
I make my complaints on mantis. This is criticism
aimed at appreciating the value of squeak.

> - build a simple infrastructure so that we can
browse any code version
> - fix the limit of the 32Mb

I do not have the experience to tackle those tasks.
Let's look forward to help from the community.
>
>There are no other choice

Not having more choices is a bug ;-) .

>
>Stef

Addendum,

In these posts I have been harsh in my critism of the
3dot9 team.  And I have not lightened it by
acknowledging the good work you and Marcus have done.
You both have labored over a year and a half to
deliver a squeak with the seeds of a lot of
opportunities for others to build on.  Overall its a
pretty successful effort. The current 7048 addresses
most of the visible problems that arose as you did
major integrations.  And by doing the work you serve a
lot of us who just wanted to use the images as they
came out.  That was true service.  I don’t want to
loose sight of that in our current discussion.

Thanks for your service.

Yours currently in critism but always in service, --
Jerome Peace.

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around
http://mail.yahoo.com 

On 26.07.2006, at 05:41, Peace Jerome wrote:

>
> Please give consideration to the formula:
> -Create a condensed source version for Squeak-6665
> (either 3.8 final or 3.9a start).
> -Provide the load scripts to produce 3.9final with all
> changes from that source.
> -And have that as a deliveralble for the squeak
> maintenence folks.
>

How would that be different from what we have now?

>
> Also it would probalbly be good to draw others into
> the pieces of this project you don’t strictly have to
> do yourself.  I suspect there are a good number of
> squeakers who have enough expericence to condense
> Squeak-6665.

Doing a "Smalltalk condenseChanges" is not hard... if it would
not be broken in 3.8, that is.

> I don’t know how many can do what you and
> stef do with the load scripts. This might be a good
> excuse to think about how to transfer that knowledge.
> I believe what I have suggested is possible.

Possible, but in what timeframe? And who would work
on it?

You might have seen that not many people care enough to
put real time or money into it... how long do you plan to wait
for whom?

> We need
> to take care of what is needed for right now within
> the present limitation. The question of what to do
> about scalability can be postponed ‘til the future.

The question if if we want to delay the release of 3.9 forever or
not. There are two possible things:
        -> postpone having 3.9
        -> postpone having a full history of 3.9

Which do we do?

     Marcus

On Wed, 26 Jul 2006 09:36:19 +0200, Marcus Denker wrote:
>
> The question if if we want to delay the release of 3.9 forever or
> not. There are two possible things:
> -> postpone having 3.9
> -> postpone having a full history of 3.9
>
> Which do we do?

Easy: the one which affects Squeak as a whole can be postponed. So rush  
out 3.9, please.

/Klaus

>      Marcus
>
>

>
> If it doesn’t scale then you have to stop at
> somepoint. The end of a development cycle is the worst
> place to stop.  The beginning of one is better.
>

But we are past the point where we could do it.
TO REPEAT: THE CHANGES FILE IS AT 25MB
*NOW*. ITS MAXIMUM IS 32MB. THIS IS NOT
FIXABLE IN BETA.

>
> Yes.  You would have a 3.9 with all changes from
> Squeak-6665. This gives a way to check on the history
> of changes in 3.9. That’s what I need. My experience
> with Doug’ 6665 with all changes is that it is a BIG
> help.  It meant being able to track down programers
> intentions and determining where and when things went
> wrong.
>

But we will reach the limit of 32MB if we build on top
of the "all changes from 3.0" version! And "All changes
from 3.9" we have.

Squeak3.8a.from3.0.changes  20.7MB
Squeak3.8.changes                   9MB
Squeak3.9b-7048.changes       24.8MB
         diff to 3.8:                               15.8 MB
====================================
Squeak3.8a.from3.0.changes + diff = 20.7 + 15.8 = 36.5

36.6 > 32 ==> We have a problem.
But it is technically impossible now, as the changes file
is limited to 32MB. so?

>
>> Because you cannot even load any decent
>> package?
>
> Huh? When do you run out of the ability to load decent
> packages?
>

because the changes file will grow to >32MB and
you get an error message.

    Marcus

Hi Marcus,

Your replys to my last couple of posts have been very
illuminating and helpful in clearing up some
misunderstandings (mine).

I think I see what can be done at this point.

To restate:

My goal:

1) In order to research fixes to bugs in squeak I
would like to have easy access to the version history
of changed methods. The images with all changes from
3.0 thru xxxx along with the changes in the current
image of 3.9 have been my main tools in doing that
research.

2) This is important but not urgent and does not need
to delay you in finalizing 3.9

3) If you have complete changes from 6665 (3.8final =
3.9a first ) in the current 3.9a 7048 (or what ever
good version beyond ) then with current resources my
research would require me to fire up an extra image
but would be doable.
 
I get from what you asked below that 7048 is
essentially equivalent to what I was trying to
suggest.
And that it is not missing any vital historical
information.

Is that a correct understanding?

**********************
**********************

4) Since it would be REALLY HELPFUL to have access to
all version/change infomation in a single search. The
problem becomes is there a simple way to do that from
where we are now.


**********************

Is 32M a limit for one change file?
Can I(or someone with more expericence) modify the
version searcher to search more than one changes file.

E.G. "Hi friendly version searcher most of the
information you need is right here in the connected
changes file but would you kindly also search file
AllChangesUpto6665.changes for versions that happened
before our time?"

That would be beautiful. Maybe then we could go back
to version 1.1 and really see how squeak developed.
That would really be educational and helpful.

Thanks again for sticking with me on this. I look
forward to your next reply.

Yours in service, -- Jerome Peace  

--- Marcus Denker <[hidden email]> wrote:

>
> On 26.07.2006, at 05:41, Peace Jerome wrote:
>
> >
Duh! That was the illuminating question.


Cheers --Jer

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around
http://mail.yahoo.com 

Ok
What would be good is to have a DB to which we could push all the  
changes
of squeak since the beginning.

> 3) If you have complete changes from 6665 (3.8final =
> 3.9a first ) in the current 3.9a 7048 (or what ever
> good version beyond ) then with current resources my
> research would require me to fire up an extra image
> but would be doable.
>
> I get from what you asked below that 7048 is
> essentially equivalent to what I was trying to
> suggest.

No because we did not started from a full history image if I remember  
correctly.

Sources are hardcoded in Smalltalk
look for at: 1 and at: 2
So this should be changed but with care.

> E.G. "Hi friendly version searcher most of the
> information you need is right here in the connected
> changes file but would you kindly also search file
> AllChangesUpto6665.changes for versions that happened
> before our time?"
>
> That would be beautiful. Maybe then we could go back
> to version 1.1 and really see how squeak developed.
> That would really be educational and helpful.

yes.
The best would be to have an external DB to manage
all the versions (in addition to changes).

At 10:37 27.07.2006, Marcus (?) wrote:
>>My goal:
>>
>>1) In order to research fixes to bugs in squeak I
>>would like to have easy access to the version history
>>of changed methods. The images with all changes from
>>3.0 thru xxxx along with the changes in the current
>>image of 3.9 have been my main tools in doing that
>>research.

Our updating scheme in the OOram project may be of interest here. (The code
was in VW):

1)
Major releases where named by a single letter. A basic release started its
life with condensed sources in source file 1. The main change file (source
file 2) was empty.
2)
Intermediate releases were named with sequence numbers, e.g., G8. The
changes from the major release were in a source file number 2.
3)
Individual programmers copied the current release with its two read-only
source files and added a sources file number 3 to hold his or her private
changes. (I found it impractical to add a third source file in Squeak; a
great pity)

My current image (I'm still playing with it from time to time. Source no. 1
is 25MB, no. 2 is 7MB, no. 3 (my personal, g09-trygve.st, is 17MB)

Every method source has a header of change notifications, e.g.,
     " 920317 trygve(4.0): Use Cursor execute, not wait. "
     " 19990125 pst (g09-1): Moved from 'end:' to be shared with subclasses. "
     " 19990125 pst (g09-1): Superfluous 'end' removed from last page. "
A designated "releasebuilder" collected all contributions. Software checked
the date sequences  for submitted methods. Branches in the modification
tree indicated conflicts and could be detected easily.  This worked well
for us in our closed community of less than a dozen programmers. ctrW
created a comment line with date and signature. The cursor left in a
position for writing additional comment. (Laziness is the mother of invention)

There has been discussions about adding extra info to the methods. One
candidate is the method's life history, automating the discipline required
by the OOram scheme. An entry could be added automatically at each compile,
giving the date, release name and author name. Compression of the entry
list could be automatic at submission time. The programmer could also be
invited to add a final comment at this time. Harvesters could esily find
conflicts between submissions.

cheers
--Trygve




--

Trygve Reenskaug      mailto: [hidden email]
Morgedalsvn. 5A       http://heim.ifi.uio.no/~trygver
N-0378 Oslo           Tel: (+47) 22 49 57 27
Norway

>stéphane ducasse ducasse at iam.unibe.ch
>Thu Jul 27 08:37:36 UTC 2006  wrote:

>What would be good is to have a DB to which we could
push all the  
>changes of squeak since the beginning.

I have problems with this approach as a solution to my
current need.

My goal: Is to have AS SOON AS POSSIBLE a way to
research fixes to bugs in squeak.

The DB project has too many risks associated with it
to be the soluton to that goal.
1) It will take resources.
2) It will take time.
3) There is no guarentee of the quality of the result
or that it will even work.
4) You have not stated the scope of the DB project
precisely enough to allow further assessment of risk
or desirability.
5) The scope even as stated here is too large for one
step.

It would be better to do the simplist thing that might
possibly work now. Learn from it and reassess the need
for an external DB.

My local disk abounds with Squeak .images and
.changes.

As a first step,
It would seem to me much easier just to teach a
version tracker or its equivalent to look for versions
in two .changes file. The local one and one of my
choosing.

The experience gained from this would inform the
larger project you suggest.


An alternate step,
1) If the proposed patch to allow 512MB change files
actually works then I would like a 70xx with all
changes from 3.0 as a research too for
maintainers/developers.

This might be even easier to get done. Especially if a
resourse other than you or Marcus could take on the
creation of the allChanges build.


I look forward to your response.

Your is service, -- Jerome Peace

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around
http://mail.yahoo.com 

>
>Squeak Maintainence and etc. ; Problem restatement.
>stéphane ducasse ducasse at iam.unibe.ch
>Thu Jul 27 08:37:36 UTC 2006 wrote:

>> My goal:
>>
>> 1) In order to research fixes to bugs in squeak I
>> would like to have easy access to the version
history
>> of changed methods. The images with all changes
from
>> 3.0 thru xxxx along with the changes in the current
>> image of 3.9 have been my main tools in doing that
>> research.
>>
>> 2) This is important but not urgent and does not
need
>> to delay you in finalizing 3.9
>
>Ok
>What would be good is to have a DB to which we could
push all the  
>changes of squeak since the beginning.



>
>> 3) If you have complete changes from 6665 (3.8final
=
>> 3.9a first ) in the current 3.9a 7048 (or what ever
>> good version beyond ) then with current resources
my
>> research would require me to fire up an extra image
>> but would be doable.
>>
>> I get from what you asked below that 7048 is
>> essentially equivalent to what I was trying to
>> suggest.
>
>No because we did not start from a full history image
if I remember  
>correctly.

Uhm. I think we’ve run into language problems again
the suggestion I was refering to was
the more recent one:

>>>> Please give consideration to the formula:
>>>> -Create a condensed source version for
Squeak-6665
>>>> (either 3.8 final or 3.9a start).
>>>> -Provide the load scripts to produce 3.9final
with all
>>>> changes from that source.
>>>> -And have that as a deliveralble for the squeak
>>>> maintenence folks.

not the first wish to have a 7048 with all changes
from 3.0. Which is what you would get from loading on
top of a full history image.

The important thing is to be able to find (in one
image or another ) the versions that lead up to the
current one. And to get the most information out of
them as to what changed; by whom; and to infer for
what purpose.

to
>> all version/change infomation in a single search.
The
>> problem becomes is there a simple way to do that
from
>> where we are now.
>>
>>
>> **********************
>>
>> Is 32M a limit for one change file?
>> Can I(or someone with more expericence) modify the
>> version searcher to search more than one changes
file.
>
>Sources are hardcoded in Smalltalk
>look for at: 1 and at: 2
>So this should be changed but with care.

Ok. So I would be possible to experiment,
>
>> E.G. "Hi friendly version searcher most of the
>> information you need is right here in the connected
>> changes file but would you kindly also search file
>> AllChangesUpto6665.changes for versions that
happened
>> before our time?"
>>
>> That would be beautiful. Maybe then we could go
back
>> to version 1.1 and really see how squeak developed.
>> That would really be educational and helpful.
>
>yes.
>The best would be to have an external DB to manage
>all the versions (in addition to changes).

But, not as a first step. (See my other post on this.)


<snip>

Yours in Service, -- Jerome Peace

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around
http://mail.yahoo.com 

On Sat, 29 Jul 2006 08:42:09 +0200, Peace Jerome wrote:
...
> The important thing is to be able to find (in one
> image or another ) the versions that lead up to the
> current one. And to get the most information out of
> them as to what changed; by whom; and to infer for
> what purpose.

I doubt that this can be achieved, at least not with the current tools  
(and .image). Simple counterexample: a method which is no longer active in  
the image is an orphan in the .sources or the .changes file. Only if the  
corresponding ChangeRecord instance is still in the image and accessible  
by a tool, then you'd be able to access it for your investigation.

Example for an orphan in .source : ThreePhaseButtonMorph methodsFor:  
'geometry' stamp: 'tk 7/1/97 09:14' #extent. This method is no longer in  
the image (was probably not distributed with a change set).

Example for an orphan in .changes : ChangeListPlus methodsFor: 'viewing  
acces' stamp: 'ar 7/15/2005 13:53' #diffedVersionContents. This method has  
two other change records, one of which should have "prior:" pointing to  
the orphan.

Attached is a script which does the following (with #7048):

1 - read all chunks from .sources and note the preamble and chunk position
2 - read all chunks from .changes and note the preamble and chunk position

This is done for class definitions (they have no preamble so the chunk  
position is taken to be := the preamble position), for class comments  
(have preambles) and for methods (have preambles). Then in a second stage  
the following is matched against the above:

3 - obtain all class comment ChangeRecords via the existing  
#scanVersionsOf: routine
4 - obtain all method ChangeRecords via the existing #changeRecordsAt:  
routine

(both routines are the exclusive methods used by browsers which need  
version information). This second stage information then overwrites the  
information obtained during the first stage, by fileIndex and filePosition  
of their sourcePointer, so that the result tells which items are linked by  
regular historical "prior:" pointers and which are not.

So when an item from steps 1-2 is not matched during steps 3-4, it must be  
an orphan.

The script allocates a new global variable SourcesAndChangesPointers by  
which the results can be accessed (and .image can be saved and analyzed  
later).

(SourcesAndChangesPointers first) are the results for the .sources file,  
(SourcesAndChangesPointers second) are the results for the .changes file.  
Both collections are instances of IdentityDiectionary.

Some datapoints obtained so far:

SourcesAndChangesPointers first size 33979 "# of non-doIt chunks (items)  
in .sources"

(SourcesAndChangesPointers first associations select: [:assoc | assoc  
value isInteger and: [assoc key ~= assoc value]]) size 13840 "# of items  
not covered by ChangeRecord routines"

(SourcesAndChangesPointers first associations select: [:assoc | assoc key  
== assoc value]) size 1509 "# of class definitions found in .sources"

(SourcesAndChangesPointers first associations reject: [:assoc | assoc  
value isInteger]) size 18630 "# of items covered by ChangeRecord routines,  
in .sources"

SourcesAndChangesPointers second size 55200 "# of non-doIt chunks (items)  
in .changes"

(SourcesAndChangesPointers second associations select: [:assoc | assoc  
value isInteger and: [assoc key ~= assoc value]]) size 11997 "# of items  
not covered by ChangeRecord routines"

(SourcesAndChangesPointers second associations select: [:assoc | assoc key  
== assoc value]) size 170 "# of class definitions found in .changes"

(SourcesAndChangesPointers second associations reject: [:assoc | assoc  
value isInteger]) size 43033 "# of items covered by ChangeRecord routines,  
in .changes"

The condition "assoc value isInteger and: [assoc key ~= assoc value]"  
denotes an item which wasn't assigned a ChangeRecord by some root in the  
image (root = class comment sourcePointer or method sourcePointer), taking  
into account all their possible history items, as computed by the  
respective routine (mentioned above). The condition "assoc key == assoc  
value" denotes class definitions found during the first stage (they have  
no source pointer in the current incarnation of the source code  
subsystem). And the exclusion "assoc value isInteger" denotes items with  
regular ChangeRecords (possible linked in history with "prior:").

More scripts could be written (but then SourcesAndChangesPointers must  
become a class), for example for the visualization of orphans and for  
viewing selections/subsets of the ChangeRecord items by using existing  
tools, like VersionsBrowser.

Disclaimer: it was carefully tested but the script may contain bugs.

>>> And that it is not missing any vital historical
>>> information.

Yes, that'd be great!

/Klaus

Klaus D. Witzel wrote:
There may be. Here is a thought that I had for a long time now: Consider
that the change history is simply one logical huge file of change
records where the .sources and .changes are mere portions of the full
"Squeak source file" that logs all the changes that were ever done as
the system evolved to your version.

Also consider that if we can describe the name of a *previous* changes
(sources) file and if we can describe where in that changes file the
previous change record for some code lives, then you could have a
situation where -in the condensed changes file of SqueakX.Y.changes- you
will find that the "previous version" of some method is to be found in
"SqueakX.Y-k.changes". In which case, merely by dropping the appropriate
changes file alongside your install you'd have the ability to browse
those previous versions - going back all the way to Squeak1.0 if you
have all those "chunks of the logical sources of Squeak" (represented by
the .changes files).

The (obvious) point being here that if we can keep the information where
previous versions ought to be located, it would be trivial to do a
condense changes for each new version - if you want history, install the
previous versions of Squeak alongside the current one and you got it.

[Disclaimer: The above obviously doesn't deal with packages etc. where
due to the current snapshot-nature of Monticello history information is
not preserved. In other words: This doesn't work for package history.].

Cheers,
   - Andreas

On Sat, 29 Jul 2006 18:16:57 +0200, Andreas Raab wrote:
Yes, like a subset of all change records.

> Also consider that if we can describe the name of a *previous* changes  
> (sources) file and if we can describe where in that changes file the  
> previous change record for some code lives, then you could have a  
> situation where -in the condensed changes file of SqueakX.Y.changes- you  
> will find that the "previous version" of some method is to be found in  
> "SqueakX.Y-k.changes".

If the information about "prior:" where accurate, always recorded and  
reliable: yes, for sure.

> In which case, merely by dropping the appropriate changes file alongside  
> your install you'd have the ability to browse those previous versions -  
> going back all the way to Squeak1.0 if you have all those "chunks of the  
> logical sources of Squeak" (represented by the .changes files).

I have converted .sources and .changes to SQLite3 (either using multiple  
tables or using mutiple db files for different releases, have choice). All  
that is needed for completing this is the correct "prior:" information  
(which seems to be rotten, see my previous message on this).

BTW: text queries, i.e. " where sourceCode like  
'%convert:%from:%to:%using:%' " are easy and fast with SQLite3 :-) This  
example finds all senders and implementors (and perhaps a bit more, can be  
further filtered), regardless of whether the method is/was ever in a  
distributed .image or not.

IMHO the "local" system can have "prior:" as filePosition but when  
travelling this should be translated to "priorStamp:". Can someone confirm  
this is sufficient, other opinions and/or experience is appreciated.

> The (obvious) point being here that if we can keep the information where  
> previous versions ought to be located, it would be trivial to do a  
> condense changes for each new version - if you want history, install the  
> previous versions of Squeak alongside the current one and you got it.

That'd be easy, I agree.

> [Disclaimer: The above obviously doesn't deal with packages etc. where  
> due to the current snapshot-nature of Monticello history information is  
> not preserved. In other words: This doesn't work for package history.].

Similiar to what Trygve described, "external" packages deserve their own  
fileIndex (or db table or file). That's all about it. If the package  
maintainer does a change, the (packageFiles at: fileIndex) gets updated.  
If a package user does a change to a package, her/his own (changeFiles at:  
fileIndex) gets updated. Can, if someone cares, be replayed on updated  
packages.

Some support for collaborative work (feedback, unaccepted and accepted  
changes) would make this an acceptable and usable system, IMO. Very small,  
very subtle, very easy steps. Not a big bang.

/Klaus

> Cheers,
>    - Andreas
>
>

Klaus D. Witzel puso en su mail :

> I have converted .sources and .changes to SQLite3 (either using multiple
> tables or using mutiple db files for different releases, have choice). All
> that is needed for completing this is the correct "prior:" information
> (which seems to be rotten, see my previous message on this).

Klaus:

I looking about SQLite3 use from Squeak , but found references to Python,
Ruby, etc.

You have any to share or point to? I need a Mac OS X version.

Very thanks.

Edgar



       
       
               
__________________________________________________
Preguntá. Respondé. Descubrí.
Todo lo que querías saber, y lo que ni imaginabas,
está en Yahoo! Respuestas (Beta).
¡Probalo ya!
http://www.yahoo.com.ar/respuestas

Edgar,

your Tiger should have SQLite pre-installed, see

- http://www.apple.com/macosx/features/unix/

And on SqMap there is the SQLite3 package (based on the SQLite package  
 from Avi) which was built and tested on Mac OS X Tiger (10.4.2). The SqMap  
entry has installation/configuration hints for Tiger.

The SQLite3 db software is most current but if needed can be recompiled  
 from source (it is FREE of copyright :)

- http://www.sqlite.org/download.html

Here I use the Win32 pre-compiled on MS$/XP. The commandline utility is  
great and can open a db in ":memory:" on my 1GB notebook :)

Have fun !

/Klaus

On Sun, 30 Jul 2006 10:53:45 +0200, Lic. Edgar J. De Cleene wrote: