Process improvement for 4.6, a new .sources file!

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
15 messages Options
Reply | Threaded
Open this post in threaded view
|

Process improvement for 4.6, a new .sources file!

Chris Muller-3
I've spent my last couple of afternoons learning about and tweaking
our condenseSources methods and process and think have come to
something that gives us the best balance between all factors:  size,
thoroughness, and convention.

  http://wiki.squeak.org/squeak/6196

Reply | Threaded
Open this post in threaded view
|

Re: Process improvement for 4.6, a new .sources file!

vaidasd
Thank you. This is useful.
Vaidas


On Mon, Mar 17, 2014 at 4:23 AM, Chris Muller <[hidden email]> wrote:
I've spent my last couple of afternoons learning about and tweaking
our condenseSources methods and process and think have come to
something that gives us the best balance between all factors:  size,
thoroughness, and convention.

  http://wiki.squeak.org/squeak/6196




Reply | Threaded
Open this post in threaded view
|

Re: Process improvement for 4.6, a new .sources file!

Hannes Hirzel
Question of clarification:

What does the current Squeak 4.5 sources file contain in terms of
methods history?

How far does it go back in time?

--Hannes

On 3/17/14, Vaidotas Didžbalis <[hidden email]> wrote:

> Thank you. This is useful.
> Vaidas
>
>
> On Mon, Mar 17, 2014 at 4:23 AM, Chris Muller <[hidden email]> wrote:
>
>> I've spent my last couple of afternoons learning about and tweaking
>> our condenseSources methods and process and think have come to
>> something that gives us the best balance between all factors:  size,
>> thoroughness, and convention.
>>
>>   http://wiki.squeak.org/squeak/6196
>>
>>
>

Reply | Threaded
Open this post in threaded view
|

Re: Process improvement for 4.6, a new .sources file!

Frank Shearar-3
On 17 March 2014 09:34, H. Hirzel <[hidden email]> wrote:
> Question of clarification:
>
> What does the current Squeak 4.5 sources file contain in terms of
> methods history?
>
> How far does it go back in time?

Unless I've completely misunderstood the purpose of the sources and
changes files, the 4.5 sources file goes back to at least 4.1.

frank

> --Hannes
>
> On 3/17/14, Vaidotas Didžbalis <[hidden email]> wrote:
>> Thank you. This is useful.
>> Vaidas
>>
>>
>> On Mon, Mar 17, 2014 at 4:23 AM, Chris Muller <[hidden email]> wrote:
>>
>>> I've spent my last couple of afternoons learning about and tweaking
>>> our condenseSources methods and process and think have come to
>>> something that gives us the best balance between all factors:  size,
>>> thoroughness, and convention.
>>>
>>>   http://wiki.squeak.org/squeak/6196
>>>
>>>
>>
>

Reply | Threaded
Open this post in threaded view
|

Re: Process improvement for 4.6, a new .sources file!

Hannes Hirzel
Chris, are you aiming at deleting the history which goes back to 4.1
and only keep the 4.5 version and the changes introduced during 4.6
development?



On 3/17/14, Frank Shearar <[hidden email]> wrote:

> On 17 March 2014 09:34, H. Hirzel <[hidden email]> wrote:
>> Question of clarification:
>>
>> What does the current Squeak 4.5 sources file contain in terms of
>> methods history?
>>
>> How far does it go back in time?
>
> Unless I've completely misunderstood the purpose of the sources and
> changes files, the 4.5 sources file goes back to at least 4.1.
>
> frank
>
>> --Hannes
>>
>> On 3/17/14, Vaidotas Didžbalis <[hidden email]> wrote:
>>> Thank you. This is useful.
>>> Vaidas
>>>
>>>
>>> On Mon, Mar 17, 2014 at 4:23 AM, Chris Muller <[hidden email]>
>>> wrote:
>>>
>>>> I've spent my last couple of afternoons learning about and tweaking
>>>> our condenseSources methods and process and think have come to
>>>> something that gives us the best balance between all factors:  size,
>>>> thoroughness, and convention.
>>>>
>>>>   http://wiki.squeak.org/squeak/6196
>>>>
>>>>
>>>
>>
>
>

Reply | Threaded
Open this post in threaded view
|

Re: Process improvement for 4.6, a new .sources file!

Chris Muller-4
Hi Hannes, yes.  And the 4.7 version would have the 4.6 and 4.7
versions of each method.  The aim is to be able to deploy a smaller
system (changes and sources) but also leave a small method version
trail _between_ the image versions.


On Mon, Mar 17, 2014 at 9:18 AM, H. Hirzel <[hidden email]> wrote:

> Chris, are you aiming at deleting the history which goes back to 4.1
> and only keep the 4.5 version and the changes introduced during 4.6
> development?
>
>
>
> On 3/17/14, Frank Shearar <[hidden email]> wrote:
>> On 17 March 2014 09:34, H. Hirzel <[hidden email]> wrote:
>>> Question of clarification:
>>>
>>> What does the current Squeak 4.5 sources file contain in terms of
>>> methods history?
>>>
>>> How far does it go back in time?
>>
>> Unless I've completely misunderstood the purpose of the sources and
>> changes files, the 4.5 sources file goes back to at least 4.1.
>>
>> frank
>>
>>> --Hannes
>>>
>>> On 3/17/14, Vaidotas Didžbalis <[hidden email]> wrote:
>>>> Thank you. This is useful.
>>>> Vaidas
>>>>
>>>>
>>>> On Mon, Mar 17, 2014 at 4:23 AM, Chris Muller <[hidden email]>
>>>> wrote:
>>>>
>>>>> I've spent my last couple of afternoons learning about and tweaking
>>>>> our condenseSources methods and process and think have come to
>>>>> something that gives us the best balance between all factors:  size,
>>>>> thoroughness, and convention.
>>>>>
>>>>>   http://wiki.squeak.org/squeak/6196
>>>>>
>>>>>
>>>>
>>>
>>
>>

Reply | Threaded
Open this post in threaded view
|

Re: Process improvement for 4.6, a new .sources file!

Hannes Hirzel
Hi Chris,

So far so good. But what would people do who still want to see the
change history back to 4.1?

Is it possible to have them load the additional history as a
separately down-loadable file in 4.6?

I assume it is, because Bob Arning is doing it back to version 1.0 in
one of his images but the question is how much effort is this?

H.

On 3/17/14, Chris Muller <[hidden email]> wrote:

> Hi Hannes, yes.  And the 4.7 version would have the 4.6 and 4.7
> versions of each method.  The aim is to be able to deploy a smaller
> system (changes and sources) but also leave a small method version
> trail _between_ the image versions.
>
>
> On Mon, Mar 17, 2014 at 9:18 AM, H. Hirzel <[hidden email]> wrote:
>> Chris, are you aiming at deleting the history which goes back to 4.1
>> and only keep the 4.5 version and the changes introduced during 4.6
>> development?
>>
>>
>>
>> On 3/17/14, Frank Shearar <[hidden email]> wrote:
>>> On 17 March 2014 09:34, H. Hirzel <[hidden email]> wrote:
>>>> Question of clarification:
>>>>
>>>> What does the current Squeak 4.5 sources file contain in terms of
>>>> methods history?
>>>>
>>>> How far does it go back in time?
>>>
>>> Unless I've completely misunderstood the purpose of the sources and
>>> changes files, the 4.5 sources file goes back to at least 4.1.
>>>
>>> frank
>>>
>>>> --Hannes
>>>>
>>>> On 3/17/14, Vaidotas Didžbalis <[hidden email]> wrote:
>>>>> Thank you. This is useful.
>>>>> Vaidas
>>>>>
>>>>>
>>>>> On Mon, Mar 17, 2014 at 4:23 AM, Chris Muller <[hidden email]>
>>>>> wrote:
>>>>>
>>>>>> I've spent my last couple of afternoons learning about and tweaking
>>>>>> our condenseSources methods and process and think have come to
>>>>>> something that gives us the best balance between all factors:  size,
>>>>>> thoroughness, and convention.
>>>>>>
>>>>>>   http://wiki.squeak.org/squeak/6196
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>>
>

Reply | Threaded
Open this post in threaded view
|

Re: Process improvement for 4.6, a new .sources file!

Chris Muller-4
I don't know whether its possible to generate a .sources file that
would contain full history, but there are options for seeing history:

  1) you can see it in the 4.5 image!  :)
  2) see http://lists.squeakfoundation.org/pipermail/squeak-dev/2013-September/173056.html
and http://lists.squeakfoundation.org/pipermail/squeak-dev/2013-November/174792.html
  3) And Bob's as you mentioned.


On Mon, Mar 17, 2014 at 10:12 AM, H. Hirzel <[hidden email]> wrote:

> Hi Chris,
>
> So far so good. But what would people do who still want to see the
> change history back to 4.1?
>
> Is it possible to have them load the additional history as a
> separately down-loadable file in 4.6?
>
> I assume it is, because Bob Arning is doing it back to version 1.0 in
> one of his images but the question is how much effort is this?
>
> H.
>
> On 3/17/14, Chris Muller <[hidden email]> wrote:
>> Hi Hannes, yes.  And the 4.7 version would have the 4.6 and 4.7
>> versions of each method.  The aim is to be able to deploy a smaller
>> system (changes and sources) but also leave a small method version
>> trail _between_ the image versions.
>>
>>
>> On Mon, Mar 17, 2014 at 9:18 AM, H. Hirzel <[hidden email]> wrote:
>>> Chris, are you aiming at deleting the history which goes back to 4.1
>>> and only keep the 4.5 version and the changes introduced during 4.6
>>> development?
>>>
>>>
>>>
>>> On 3/17/14, Frank Shearar <[hidden email]> wrote:
>>>> On 17 March 2014 09:34, H. Hirzel <[hidden email]> wrote:
>>>>> Question of clarification:
>>>>>
>>>>> What does the current Squeak 4.5 sources file contain in terms of
>>>>> methods history?
>>>>>
>>>>> How far does it go back in time?
>>>>
>>>> Unless I've completely misunderstood the purpose of the sources and
>>>> changes files, the 4.5 sources file goes back to at least 4.1.
>>>>
>>>> frank
>>>>
>>>>> --Hannes
>>>>>
>>>>> On 3/17/14, Vaidotas Didžbalis <[hidden email]> wrote:
>>>>>> Thank you. This is useful.
>>>>>> Vaidas
>>>>>>
>>>>>>
>>>>>> On Mon, Mar 17, 2014 at 4:23 AM, Chris Muller <[hidden email]>
>>>>>> wrote:
>>>>>>
>>>>>>> I've spent my last couple of afternoons learning about and tweaking
>>>>>>> our condenseSources methods and process and think have come to
>>>>>>> something that gives us the best balance between all factors:  size,
>>>>>>> thoroughness, and convention.
>>>>>>>
>>>>>>>   http://wiki.squeak.org/squeak/6196
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>

Reply | Threaded
Open this post in threaded view
|

Re: Process improvement for 4.6, a new .sources file!

timrowledge
In reply to this post by Hannes Hirzel

On 17-03-2014, at 8:12 AM, H. Hirzel <[hidden email]> wrote:

> Hi Chris,
>
> So far so good. But what would people do who still want to see the
> change history back to 4.1?
>
> Is it possible to have them load the additional history as a
> separately down-loadable file in 4.6?
>
> I assume it is, because Bob Arning is doing it back to version 1.0 in
> one of his images but the question is how much effort is this?

The .sources file should have the sources for each and every method that was in the image on the last occasion when ‘condenseSources’ was run. In ‘the old days’ that meant one and only one version of each method. The general idea was for each release you did a condense in order to get a single large .sources file and a virtually empty .changes file.

At some point an ‘appendChanges’ was added so that you could copy the latest version of any method source in the .changes to the end of the existing .sources - this makes for a single large[r] file and the empty .changes but does give you a single step of history for any of the methods changed since the last condenseSources.

I don’t know, but anticipate that a second ‘appendChanges’ would just add any subsequently changed methods’ source, which makes for a gradually growing .sources file that might be useful or might be ludicrously large, depending upon how much code gets changed over time.

I’m not a big fan of the append approach in general. To maintain a history I much prefer the online server that can provide that info - as provided both by Chris Muller’s MC History (so much cooler than MC Hammer) and Bob’s site that I can’t seem to find right now. It would be nice to have the changes browser and other history sensitive tools able to access them on request - shouldn’t happen by default since it takes time.

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Strange OpCodes: BA: Branch Approximate



Reply | Threaded
Open this post in threaded view
|

Re: Process improvement for 4.6, a new .sources file!

Chris Muller-3
> I don’t know, but anticipate that a second ‘appendChanges’ would just add any subsequently changed methods’ source, which makes for a gradually growing .sources file that might be useful or might be ludicrously large, depending upon how much code gets changed over time.

Just to clarify, there is no plan to do a second appendChanges.  A new
sources is rebuilt anew every release, so the sources won't grow
"gradually".  Just once, incrementally.  We're talking about probably
a couple of MB, not "ludicrous".  And, if we are successful in
reducing the size of the system, that will effect to reduce the size
of .sources anyway.

For just a couple of MB (I think), we can connect Squeak's
release-to-release.  The external tools don't do that.  Is anyone
besides me actually using them?

Reply | Threaded
Open this post in threaded view
|

Re: Process improvement for 4.6, a new .sources file!

Eliot Miranda-2
In reply to this post by Chris Muller-4



On Mon, Mar 17, 2014 at 8:28 AM, Chris Muller <[hidden email]> wrote:
I don't know whether its possible to generate a .sources file that
would contain full history,

_yes it is_.  The only issue in preserving history is in pruning branches.  There is no point preserving versions that are not ancestors of current versions.  And that's an easy process.  I have code that does this and Camillo just rewrote source condensing for Pharo and claims its much more comprehensible, so taking his code and getting it to preserve history while pruning could be another route.

 
but there are options for seeing history:

  1) you can see it in the 4.5 image!  :)
  2) see http://lists.squeakfoundation.org/pipermail/squeak-dev/2013-September/173056.html
and http://lists.squeakfoundation.org/pipermail/squeak-dev/2013-November/174792.html
  3) And Bob's as you mentioned.


On Mon, Mar 17, 2014 at 10:12 AM, H. Hirzel <[hidden email]> wrote:
> Hi Chris,
>
> So far so good. But what would people do who still want to see the
> change history back to 4.1?
>
> Is it possible to have them load the additional history as a
> separately down-loadable file in 4.6?
>
> I assume it is, because Bob Arning is doing it back to version 1.0 in
> one of his images but the question is how much effort is this?
>
> H.
>
> On 3/17/14, Chris Muller <[hidden email]> wrote:
>> Hi Hannes, yes.  And the 4.7 version would have the 4.6 and 4.7
>> versions of each method.  The aim is to be able to deploy a smaller
>> system (changes and sources) but also leave a small method version
>> trail _between_ the image versions.
>>
>>
>> On Mon, Mar 17, 2014 at 9:18 AM, H. Hirzel <[hidden email]> wrote:
>>> Chris, are you aiming at deleting the history which goes back to 4.1
>>> and only keep the 4.5 version and the changes introduced during 4.6
>>> development?
>>>
>>>
>>>
>>> On 3/17/14, Frank Shearar <[hidden email]> wrote:
>>>> On 17 March 2014 09:34, H. Hirzel <[hidden email]> wrote:
>>>>> Question of clarification:
>>>>>
>>>>> What does the current Squeak 4.5 sources file contain in terms of
>>>>> methods history?
>>>>>
>>>>> How far does it go back in time?
>>>>
>>>> Unless I've completely misunderstood the purpose of the sources and
>>>> changes files, the 4.5 sources file goes back to at least 4.1.
>>>>
>>>> frank
>>>>
>>>>> --Hannes
>>>>>
>>>>> On 3/17/14, Vaidotas Didžbalis <[hidden email]> wrote:
>>>>>> Thank you. This is useful.
>>>>>> Vaidas
>>>>>>
>>>>>>
>>>>>> On Mon, Mar 17, 2014 at 4:23 AM, Chris Muller <[hidden email]>
>>>>>> wrote:
>>>>>>
>>>>>>> I've spent my last couple of afternoons learning about and tweaking
>>>>>>> our condenseSources methods and process and think have come to
>>>>>>> something that gives us the best balance between all factors:  size,
>>>>>>> thoroughness, and convention.
>>>>>>>
>>>>>>>   http://wiki.squeak.org/squeak/6196
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>




--
best,
Eliot


Reply | Threaded
Open this post in threaded view
|

Re: Process improvement for 4.6, a new .sources file!

Chris Muller-3
On Tue, Mar 18, 2014 at 2:47 PM, Eliot Miranda <[hidden email]> wrote:

>
>
>
> On Mon, Mar 17, 2014 at 8:28 AM, Chris Muller <[hidden email]> wrote:
>>
>> I don't know whether its possible to generate a .sources file that
>> would contain full history,
>
>
> _yes it is_.  The only issue in preserving history is in pruning branches.
> There is no point preserving versions that are not ancestors of current
> versions.  And that's an easy process.  I have code that does this and
> Camillo just rewrote source condensing for Pharo and claims its much more
> comprehensible, so taking his code and getting it to preserve history while
> pruning could be another route.

How does your or Camillo's code work?  How could it possibly know what
version of, say, Dictionary>>#at:put: was released to official Squeak
2.8?  Are you parsing into some cold 2.8 image file from a 4.5 image?

Even if you could identify all the CM's you still don't have the
source for them.  You'd have to then go looking into the cold image
file, identify which .sources file its using, find the source-pointer
for that method, THEN go into that sources file to extract it..

I just don't believe you or Camillo are attempting something this
crazy -- hell just setting up all the images and sources would be a
major pain.  But I see no other way to accomplish what Hannes was
referring to as "full history".  Tell me how it works, show me the
code and how to use it, else I call bull-shit.  ;)

Reply | Threaded
Open this post in threaded view
|

Re: Process improvement for 4.6, a new .sources file!

Eliot Miranda-2



On Tue, Mar 18, 2014 at 1:07 PM, Chris Muller <[hidden email]> wrote:
On Tue, Mar 18, 2014 at 2:47 PM, Eliot Miranda <[hidden email]> wrote:
>
>
>
> On Mon, Mar 17, 2014 at 8:28 AM, Chris Muller <[hidden email]> wrote:
>>
>> I don't know whether its possible to generate a .sources file that
>> would contain full history,
>
>
> _yes it is_.  The only issue in preserving history is in pruning branches.
> There is no point preserving versions that are not ancestors of current
> versions.  And that's an easy process.  I have code that does this and
> Camillo just rewrote source condensing for Pharo and claims its much more
> comprehensible, so taking his code and getting it to preserve history while
> pruning could be another route.

How does your or Camillo's code work?  How could it possibly know what
version of, say, Dictionary>>#at:put: was released to official Squeak
2.8?  Are you parsing into some cold 2.8 image file from a 4.5 image?

The code merely collects all available versions of each method, prunes the versions which have no descendents and writes all these methods to the sources file.  [I don't know how Camillo;s code works].  So this won't magically retrieve older versions, but it *will* preserve the history we have since the writing of the 4.1 sources file.


Even if you could identify all the CM's you still don't have the
source for them.  You'd have to then go looking into the cold image
file, identify which .sources file its using, find the source-pointer
for that method, THEN go into that sources file to extract it..

That's not what it does.  What it does is condense preserving available history.  What we have no is condense, losing all available history.  As we accumulate more history so preserving extant history becomes more and more valuable, no?

I just don't believe you or Camillo are attempting something this
crazy -- hell just setting up all the images and sources would be a
major pain.  But I see no other way to accomplish what Hannes was
referring to as "full history".  Tell me how it works, show me the
code and how to use it, else I call bull-shit.  ;)

But that's also quite possible and not that hard either.  A CHange List is a tool which can accomplish most of this.  It is a few days work.  But as I say, we can much more easily condense while preserving extant history.  Am I clearer now?
-- 
best,
Eliot


Reply | Threaded
Open this post in threaded view
|

Re: Process improvement for 4.6, a new .sources file!

Chris Muller-4
>> >> I don't know whether its possible to generate a .sources file that
>> >> would contain full history,
>> >
>> >
>> > _yes it is_.  The only issue in preserving history is in pruning
>> > branches.
>> > There is no point preserving versions that are not ancestors of current
>> > versions.  And that's an easy process.  I have code that does this and
>> > Camillo just rewrote source condensing for Pharo and claims its much
>> > more
>> > comprehensible, so taking his code and getting it to preserve history
>> > while
>> > pruning could be another route.
>>
>> How does your or Camillo's code work?  How could it possibly know what
>> version of, say, Dictionary>>#at:put: was released to official Squeak
>> 2.8?  Are you parsing into some cold 2.8 image file from a 4.5 image?
>
>
> The code merely collects all available versions of each method, prunes the
> versions which have no descendents and writes all these methods to the
> sources file.  [I don't know how Camillo;s code works].  So this won't
> magically retrieve older versions, but it *will* preserve the history we
> have since the writing of the 4.1 sources file.

Something I'm still trying to understand; what are the ones with "no
descendents"?  Wouldn't that just be the most-recent version?  Ah, you
mean all "the duplicates"..?

>> Even if you could identify all the CM's you still don't have the
>> source for them.  You'd have to then go looking into the cold image
>> file, identify which .sources file its using, find the source-pointer
>> for that method, THEN go into that sources file to extract it..
>
>
> That's not what it does.  What it does is condense preserving available
> history.  What we have no is condense, losing all available history.  As we
> accumulate more history so preserving extant history becomes more and more
> valuable, no?
>
>> I just don't believe you or Camillo are attempting something this
>> crazy -- hell just setting up all the images and sources would be a
>> major pain.  But I see no other way to accomplish what Hannes was
>> referring to as "full history".  Tell me how it works, show me the
>> code and how to use it, else I call bull-shit.  ;)
>
>
> But that's also quite possible and not that hard either.  A CHange List is a
> tool which can accomplish most of this.  It is a few days work.  But as I
> say, we can much more easily condense while preserving extant history.  Am I
> clearer now?

Not really.  But, I, for one, am for keeping all history (sans
duplicates).  I simply thought there was movement in the community to
want to use #condenseSources for deployment with no preservation of
anything.  Anyone concerned about size can easily condense their own
sources, but us who like full history cannot get that back easily if
we deploy condensedSources.  So, I like your idea better.

Should your code be run at the beginning of release cycle or the end?

Reply | Threaded
Open this post in threaded view
|

Re: Process improvement for 4.6, a new .sources file!

Hannes Hirzel
On 3/20/14, Chris Muller <[hidden email]> wrote:

>
>  Anyone concerned about size can easily condense their own
> sources,

Yes, and the way how to do it might be included in the 'useful
expressions' workspace.

--Hannes