Brave New World

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
33 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Brave New World

keith1y
> BTW, now that we've agreed that it doesn't matter, do you concede  
> the point that Monticello is substantially like git/mercurial/bazaar  
> from a technical perspective, even if we're not using it in the way  
> that the Mercurial folks advocate?  ;-)

No because if it were true, firstly Gofer would never have been  
developed.

We are not talking about the ability to work locally if you want to.  
We are talking about having all of the history of (in my case) some  
60+ packages from different sources, locally, and providing a non-
manual resync back to base when you get back on line again.

You can use "MC1.5's copy all (forked)", to grab the whole history of  
a package and keep it locally. if you want to, but that isnt really  
how the tool is designed to be used.

Keith




Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Brave New World

Josh Gargus
On Jan 24, 2010, at 12:00 PM, keith wrote:

>> BTW, now that we've agreed that it doesn't matter, do you concede the point that Monticello is substantially like git/mercurial/bazaar from a technical perspective, even if we're not using it in the way that the Mercurial folks advocate?  ;-)
>
> No because if it were true, firstly Gofer would never have been developed.


(looking up Gofer, which I was unfamiliar with... OK, got it)


>
> We are not talking about the ability to work locally if you want to. We are talking about having all of the history of (in my case) some 60+ packages from different sources, locally, and providing a non-manual resync back to base when you get back on line again.


OK, I'm completely lost.  What the hell are we talking about here?

I thought that we are talking about how Bob/Sake/Packages are a first attempt at a tool chain that can support the "Mercurial" approach to distributed development.  Furthermore, we agreed that these tools aren't perfect, and that we would therefore make an effort to evaluate the ideas that the tools are an expression of.

Right?

Since we just established this one email exchange ago, and since I'm sure that you're good enough to extend to Monticello the same exemption from purely technical nitpicking, then I must conclude that the cited differences between Monticello and Mercurial are relevant with respect to the topic as I just framed it above.  

Right?

So please explain to me...

Why is having the entire history of packages available locally *fundamentally* necessary for the development process that you're describing (as opposed to for your convenience while developing on the train)?

Why is providing a non-manual resync back to base *fundamentally* necessary for the development process that you're describing (as opposed to your convenience while developing on the train)?

I submit that the online/offline distinction is a diversion.  It is orthogonal to the core question, and furthermore Monticello could be easily extended to add this functionality if required.  If this is not true, please explain why, because I do not understand.

Leaving aside the online/offline distinction (unless you can explain why we shouldn't), I ask again: do you concede the point that Monticello is substantially like git/mercurial/bazaar from a technical perspective, even if we're not using it in the way that the Mercurial folks advocate?


Cheers,
Josh



>
> You can use "MC1.5's copy all (forked)", to grab the whole history of a package and keep it locally. if you want to, but that isnt really how the tool is designed to be used.
>
> Keith
>
>
>
>


Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Brave New World

keith1y
In reply to this post by Josh Gargus
I see what you're saying.

I started to write responses to a couple of other posts, but I realize that there are still things that I don't understand about your proposal.  Please continue to help me understand.

Let's imagine that 3.10.2 just came out, and that we have your process in place.  I'm assuming that we can equate 3.10.2 to A.  The important part that I'm missing is that I don't know the time interval between A and A.2.  Is it on the order of a week or two, or 6 months?  Once I know the answer, I'll have more to say.

Given that once A is out (a fixed point), work begins on migrating innovations in progress, to load into A, and fixes not yet integrated, to test them against A.

The automated script generating and testing "Ax-stable-alpha-test" image candidates, runs every (?)  time a bug fix status is changed from "testing" to "passed" in mantis. So as soon as A is released, this script is switched to start loading and testing fixes against it.

Practically speaking, allowing for a 2 week break after the release to let the dust settle, a serious go at producing the next release candidate A.2-stable-beta should be made within a month and leaving a couple of weeks to finalise things. You are looking at a new fully tested release A2-stable (fixed point) every 4-6 weeks.

Also, you often refer to the codebase that you're developing against A that you will eventually want to move forward to whatever the latest is (maybe A.2, or maybe A.10).  How often do you envision porting your packages forward to the latest?  Every week or two?  Every 6 months?   Does it depend on the specific developer/codebase?

There are two builds happening simultaneously, A+1-unstable, (unstable builds on an image loading mantis fixes status "testing") and A+1-stable (builds on fixes status "passed"), and you can add your own, like A+1-stable+withnamespaces, A+1-unstable+closures etc.

So package developers can see what is coming, by checking the results of the "developer" image, or the "web" image that is built upon A+1-unstable.

The new A2.3.4 etc, are automatically used as a basis for building all derived images, such as the developer image, the web image, fun, and my own working image build etc. So it is quickly obvious when a package is failing to keep up, or a kernel patch is breaking things for everyone.

Given that the packages are at A.x-1, when you release A.x, for the tests to pass, you just need a version of the package that is compatible with two releases, the newest and the previous one to ensure a continuous migration path for everyone.

(BTW, I'm curious, can you cite an example of a successful project that uses this approach?  If you can't, that doesn't automatically disqualify your idea (someone has to be first), but I'm still curious.  Please, just a short yes or no, with no loss-of-face or giving-up-ground implied in a negative answer).
 
It's what I do every time I build an image to use, I have been using this process for several years, because squeak releases are few and far between.

Task 1. download the old release (A.0)
Task 2. load the tested stable kernel fixes that I know I need. (Either from mantis, by number, or from a MC package that I use to maintain them, "Kernel-Extensions")
Task 3. upgrade all the externally managed packages that the release has out of date versions of.
Task 4. load all the packages that provide my basic tools
Task 5. unload a few things.
Task 6. reorganise stuff to make things a bit tidier.
Task 7. (optionally) Save everything to monticello.

This was automated, and I save my actual working base image (A.1) that then build my product on, starting with the developer tools (optional), seaside, pier, rio, beach and magma.

Given that this process, is how I move an ageing release to be an image that is good enough to actually start working with. And I do this every month or two, and for every new client. I figured that the same process would be able to provide the monthly iterations, such that A.1 actually gets saved as 3.11.

You should cut people some slack,

Sorry but I was on my holiday when this all kicked off. Where was the slack for me?

This discussion all happened 3 years ago amongst members of the release team, and in the preparation of the proposal with the board. For 3.10 Everyone that was interested in the release was on the release-team list. Very few discussions were held on squeak-dev. 

because you have been anything but consistent on this point over the past months.  You're focusing on the abstract principles now, but at other times you've said that Bob/Mantis is good to go, and that we should start using it immediately.

Mantis, has been used for collating fixes for years. It is good to go. Automated loading of mantis fixes is provided by InstallerMantis. You can do it yourself using code along the lines of (I dont have an image open)

(Installer mantis allBugs select: [ :ea | ea status = "Testing" ]) do: #ensureFix.

We already had more fixes marked for integration than I could handle, in a single iteration. Effort would be better served packaging other innovations ready to be cleanly loadable into 3.10, such as closures. 

I thought the board understood the abstract principles, since we discussed them in person with some people, and we had to write a proposal etc.

Having said that, I'm comfortable with agreeing that criticisms of Mantis/Bob (i.e. the current concrete implementation)

are out-of-bounds while we consider the abstract principles involved (of course, if we get to the point where you convince the community that you're correct in principle, then criticizing Mantis/Bob is back on the table as we decide *how* to implement your ideas).

OK, gotcha.  Are we ready to proceed? (did I manage to avoid saying anything that you violently disagree with?)

you did ok.... would you like to run for the Board?

Keith



Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Brave New World

Josh Gargus

On Jan 24, 2010, at 1:09 PM, keith wrote:

I see what you're saying.

I started to write responses to a couple of other posts, but I realize that there are still things that I don't understand about your proposal.  Please continue to help me understand.

Let's imagine that 3.10.2 just came out, and that we have your process in place.  I'm assuming that we can equate 3.10.2 to A.  The important part that I'm missing is that I don't know the time interval between A and A.2.  Is it on the order of a week or two, or 6 months?  Once I know the answer, I'll have more to say.

Given that once A is out (a fixed point), work begins on migrating innovations in progress, to load into A, and fixes not yet integrated, to test them against A.

The automated script generating and testing "Ax-stable-alpha-test" image candidates, runs every (?)  time a bug fix status is changed from "testing" to "passed" in mantis. So as soon as A is released, this script is switched to start loading and testing fixes against it.

Practically speaking, allowing for a 2 week break after the release to let the dust settle, a serious go at producing the next release candidate A.2-stable-beta should be made within a month and leaving a couple of weeks to finalise things. You are looking at a new fully tested release A2-stable (fixed point) every 4-6 weeks.


Thanks...



Also, you often refer to the codebase that you're developing against A that you will eventually want to move forward to whatever the latest is (maybe A.2, or maybe A.10).  How often do you envision porting your packages forward to the latest?  Every week or two?  Every 6 months?   Does it depend on the specific developer/codebase?

There are two builds happening simultaneously, A+1-unstable, (unstable builds on an image loading mantis fixes status "testing") and A+1-stable (builds on fixes status "passed"), and you can add your own, like A+1-stable+withnamespaces, A+1-unstable+closures etc.

So package developers can see what is coming, by checking the results of the "developer" image, or the "web" image that is built upon A+1-unstable.

The new A2.3.4 etc, are automatically used as a basis for building all derived images, such as the developer image, the web image, fun, and my own working image build etc. So it is quickly obvious when a package is failing to keep up, or a kernel patch is breaking things for everyone.

Given that the packages are at A.x-1, when you release A.x, for the tests to pass, you just need a version of the package that is compatible with two releases, the newest and the previous one to ensure a continuous migration path for everyone.


... and thanks again.

One thing that strikes me about this is that it is agnostic about how the fixed-point images are built.  From the point of view of the application codebase that you are supporting, it will work  just as well (more or less) to use your scripts to automatically load/test packages into a released trunk image (assuming that the trunk images are released every couple of weeks).

I'm quite sure that this is basically true, right?  I'm also quite sure that you'll be quick to point out ways that it isn't!  Please do, I'm sure that I'm missing a thing or two.




(BTW, I'm curious, can you cite an example of a successful project that uses this approach?  If you can't, that doesn't automatically disqualify your idea (someone has to be first), but I'm still curious.  Please, just a short yes or no, with no loss-of-face or giving-up-ground implied in a negative answer).
 
It's what I do every time I build an image to use, I have been using this process for several years, because squeak releases are few and far between.


Thanks, I know.  I should have been more specific; I was wondering about multi-developer teams using this approach.


You should cut people some slack,

Sorry but I was on my holiday when this all kicked off. Where was the slack for me?

This discussion all happened 3 years ago amongst members of the release team, and in the preparation of the proposal with the board. For 3.10 Everyone that was interested in the release was on the release-team list. Very few discussions were held on squeak-dev. 

because you have been anything but consistent on this point over the past months.  You're focusing on the abstract principles now, but at other times you've said that Bob/Mantis is good to go, and that we should start using it immediately.

Mantis, has been used for collating fixes for years. It is good to go. Automated loading of mantis fixes is provided by InstallerMantis. You can do it yourself using code along the lines of (I dont have an image open)

(Installer mantis allBugs select: [ :ea | ea status = "Testing" ]) do: #ensureFix.

We already had more fixes marked for integration than I could handle, in a single iteration. Effort would be better served packaging other innovations ready to be cleanly loadable into 3.10, such as closures. 

I thought the board understood the abstract principles, since we discussed them in person with some people, and we had to write a proposal etc.


So, to reiterate, you're saying that earlier you were under the impression that the abstract principles were already understood, and that you didn't need to keep repeating them.  But more recently (perhaps especially in the last few days) it became apparent to you that people were criticizing the technology rather than the ideas?  Fair enough.  FWIW, I'm glad that you decided to be explicit that you want to talk about the principles rather than the technology... the communication seems to work much better for some reason.  

Cheers,
Josh




Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Brave New World

Frank Shearar
In reply to this post by keith1y
keith wrote:

>> BTW, now that we've agreed that it doesn't matter, do you concede the
>> point that Monticello is substantially like git/mercurial/bazaar from
>> a technical perspective, even if we're not using it in the way that
>> the Mercurial folks advocate?  ;-)
>
> No because if it were true, firstly Gofer would never have been developed.
>
> We are not talking about the ability to work locally if you want to. We
> are talking about having all of the history of (in my case) some 60+
> packages from different sources, locally, and providing a non-manual
> resync back to base when you get back on line again.
>
> You can use "MC1.5's copy all (forked)", to grab the whole history of a
> package and keep it locally. if you want to, but that isnt really how
> the tool is designed to be used.


Keith's right: a single MCD file is like a single Git commit. It
contains what changed, and tells you the parent on which it was based.

But "git clone" does more: it pulls all the commits - all the MCD files
- in one go.

frank

Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Brave New World

keith1y
In reply to this post by Josh Gargus

... and thanks again.

One thing that strikes me about this is that it is agnostic about how the fixed-point images are built.

The fixed point images need to be built, by a process which applies to the previous fixed point.

So that a fork which was made from the previous fixed point, can cherry pick from, and apply that process too.

The more that separate forks pick ideas from other forks, the more they have in common.

Basically to use the knowledge that is in trunk, you will have to analyse it and sort it into separate useful innovations and ideas, and publish them as separate changesets.

Ironically, most of those innovations could have started of as separate changesets in the first place. Am I making sense?


Keith



Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Brave New World

Josh Gargus

On Jan 25, 2010, at 8:31 AM, keith wrote:


... and thanks again.

One thing that strikes me about this is that it is agnostic about how the fixed-point images are built.

The fixed point images need to be built, by a process which applies to the previous fixed point.

So that a fork which was made from the previous fixed point, can cherry pick from, and apply that process too.

The more that separate forks pick ideas from other forks, the more they have in common.

Basically to use the knowledge that is in trunk, you will have to analyse it and sort it into separate useful innovations and ideas, and publish them as separate changesets.

Ironically, most of those innovations could have started of as separate changesets in the first place. Am I making sense?


Yes, I think so.

I think that the problem here is that we're talking about at least two things.  One is the ability to use improvements to one fork in another fork.  Another is the ability to move an existing application code-base forward to the newest fixed-point release.

I'm not saying anything at all about the first one here.  In particular, I'm not arguing that the process used to create images doesn't matter for cherry-picking packages between images.

I contend that for the latter, it does not matter how the fixed-point image was created, just that you continuously build and test your application against frequent releases, and work with the community to quickly fix any compatibility issues that arise.  Isn't this true?

Thanks,
Josh







Keith





Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Brave New World

keith1y
I contend that for the latter, it does not matter how the fixed-point image was created, just that you continuously build and test your application against frequent releases, and work with the community to quickly fix any compatibility issues that arise.  Isn't this true?

Thanks,
Josh

It would be lovely, but it just doesn't happen. Ask etoys, cobalt, sophie, gjallar. They all started large projects and did not find the time or the effort to track subsequent squeak releases. One problem is that individual releases were too revolutionary for many, trying to to do much in one go,  this can be helped by release early and often. More than that the release teams didn't take any responsibility for the migration path.  (e.g. Smalltalk -> SmalltalkImage)

This same problem occurs at a package level too. I am stuck about 150 (or more now) revisions behind in Pier, because my modifications to pier (which I do keep in a separate package), are effectively (but accidentally) a fork. In trying to solve the problem, the more I see the tools, i.e. Monticello as part of the problem. However at present the solution alludes me.

K.


Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Brave New World

Josh Gargus

On Jan 25, 2010, at 5:27 PM, keith wrote:

I contend that for the latter, it does not matter how the fixed-point image was created, just that you continuously build and test your application against frequent releases, and work with the community to quickly fix any compatibility issues that arise.  Isn't this true?

Thanks,
Josh

It would be lovely, but it just doesn't happen. Ask etoys, cobalt, sophie, gjallar. They all started large projects and did not find the time or the effort to track subsequent squeak releases. One problem is that individual releases were too revolutionary for many, trying to to do much in one go,  this can be helped by release early and often.


Right, I completely agree with this.  That's a good think about bi-monthly trunk images, no?


More than that the release teams didn't take any responsibility for the migration path.  (e.g. Smalltalk -> SmalltalkImage)


Please, don't get me started on SmalltalkImage.  :-)

Actually, I'm glad you mentioned it, because it highlights how the attitude of current contributors has evolved (you can trust me on this, or take an honest look at the discussions over the past months).  Making a similar change (i.e. one that breaks existing code with no demonstrable benefit except being "cleaner") would be strongly frowned upon today.  In fact, there is even skepticism about changes that don't break anything, but are only cosmetic improvements.  The reason for this change in attitude is precisely because of a concern for the migration path.  Yet another example where things might not be as bad as you've convinced yourself that they are :-)

BTW, I don't mean to disparage the efforts of the 3.9 team.  I think that SmalltalkImage was an ugly mistake, but I appreciate all of the energy they put into moving Squeak forward... and sometimes sideways ;-)

Cheers,
Josh




This same problem occurs at a package level too. I am stuck about 150 (or more now) revisions behind in Pier, because my modifications to pier (which I do keep in a separate package), are effectively (but accidentally) a fork. In trying to solve the problem, the more I see the tools, i.e. Monticello as part of the problem. However at present the solution alludes me.

K.




Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Brave New World

garduino
In reply to this post by garduino
2010/1/23 Germán Arduino <[hidden email]>:

> Will check, thanks Mariano!
>
>
> 2010/1/23 Mariano Martinez Peck <[hidden email]>:
>>
>>
>> 2010/1/23 Levente Uzonyi <[hidden email]>
>>>
>>> On Sat, 23 Jan 2010, Mariano Martinez Peck wrote:
>>>
>>>> On Sat, Jan 23, 2010 at 4:38 PM, Germán Arduino <[hidden email]>
>>>> wrote:
>>>>
>>>>> Hi:
>>>>>
>>>>> Without excluding nothing, I think that Minimal deserve a bit of
>>>>> attention. I saw it running latests version of Pier with no problems.
>>>>>
>>>>> I'm thinking in use it to Pier customers, because in shared servers
>>>>> the sizes of image is very important.
>>>>>
>>>>
>>>>
>>>> The latest PharoCore 1.0 is also 7.8 MB after ScriptLoader new
>>>> cleanUpForRelease, where Seaside and Pier can run perfectly also.
>>>
>>> Seems to be 10.7 MB.
>>>
>>
>> Ups...sorry I mean ScriptLoader new cleanUpForProduction.    not
>> ScriptLoader new cleanUpForRelease.
>>
>>
>>
>>>
>>> Levente
>>>
>>>>
>>>>
>>>>
>>>
>>>
>>>
>>
>>
>>
>>
>>
>
>
>
> --
> =================================================
> Germán S. Arduino  <gsa @ arsol.net>   Twitter: garduino
> Arduino Software & Web Hosting   http://www.arduinosoftware.com
> PasswordsPro  http://www.passwordspro.com
> =================================================
>



--
=================================================
Germán S. Arduino  <gsa @ arsol.net>   Twitter: garduino
Arduino Software & Web Hosting   http://www.arduinosoftware.com
PasswordsPro  http://www.passwordspro.com
=================================================

Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Brave New World

garduino
In reply to this post by Mariano Martinez Peck
2010/1/23 Mariano Martinez Peck <[hidden email]>:

>
>
> 2010/1/23 Levente Uzonyi <[hidden email]>
>>
>> On Sat, 23 Jan 2010, Mariano Martinez Peck wrote:
>>
>>> On Sat, Jan 23, 2010 at 4:38 PM, Germán Arduino <[hidden email]>
>>> wrote:
>>>
>>>> Hi:
>>>>
>>>> Without excluding nothing, I think that Minimal deserve a bit of
>>>> attention. I saw it running latests version of Pier with no problems.
>>>>
>>>> I'm thinking in use it to Pier customers, because in shared servers
>>>> the sizes of image is very important.
>>>>
>>>
>>>
>>> The latest PharoCore 1.0 is also 7.8 MB after ScriptLoader new
>>> cleanUpForRelease, where Seaside and Pier can run perfectly also.
>>
>> Seems to be 10.7 MB.
>>
>
> Ups...sorry I mean ScriptLoader new cleanUpForProduction.    not
> ScriptLoader new cleanUpForRelease.
>
>

I tried the ScripLoader new cleanUpForRelease on
Pharo1.0-10508-rc2dev10.01.2 and don't think
to work.

I'm missing something?

Cheers.
Germán.

Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Brave New World

Levente Uzonyi-2
On Tue, 26 Jan 2010, Germán Arduino wrote:

> 2010/1/23 Mariano Martinez Peck <[hidden email]>:
>>
>>
>> 2010/1/23 Levente Uzonyi <[hidden email]>
>>>
>>> On Sat, 23 Jan 2010, Mariano Martinez Peck wrote:
>>>
>>>> On Sat, Jan 23, 2010 at 4:38 PM, Germán Arduino <[hidden email]>
>>>> wrote:
>>>>
>>>>> Hi:
>>>>>
>>>>> Without excluding nothing, I think that Minimal deserve a bit of
>>>>> attention. I saw it running latests version of Pier with no problems.
>>>>>
>>>>> I'm thinking in use it to Pier customers, because in shared servers
>>>>> the sizes of image is very important.
>>>>>
>>>>
>>>>
>>>> The latest PharoCore 1.0 is also 7.8 MB after ScriptLoader new
>>>> cleanUpForRelease, where Seaside and Pier can run perfectly also.
>>>
>>> Seems to be 10.7 MB.
>>>
>>
>> Ups...sorry I mean ScriptLoader new cleanUpForProduction.    not
>> ScriptLoader new cleanUpForRelease.
>>
>>
>
> I tried the ScripLoader new cleanUpForRelease on
> Pharo1.0-10508-rc2dev10.01.2 and don't think
> to work.
>
> I'm missing something?
Yes, try cleanUpForProduction.


Levente

>
> Cheers.
> Germán.
>
>

Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Brave New World

garduino
oppsss my fault, sorry, and thanks Levente.

11,3MB on rc2.

Cheers.
Germàn.

2010/1/26 Levente Uzonyi <[hidden email]>:

> On Tue, 26 Jan 2010, Germán Arduino wrote:
>
>> 2010/1/23 Mariano Martinez Peck <[hidden email]>:
>>>
>>>
>>> 2010/1/23 Levente Uzonyi <[hidden email]>
>>>>
>>>> On Sat, 23 Jan 2010, Mariano Martinez Peck wrote:
>>>>
>>>>> On Sat, Jan 23, 2010 at 4:38 PM, Germán Arduino <[hidden email]>
>>>>> wrote:
>>>>>
>>>>>> Hi:
>>>>>>
>>>>>> Without excluding nothing, I think that Minimal deserve a bit of
>>>>>> attention. I saw it running latests version of Pier with no problems.
>>>>>>
>>>>>> I'm thinking in use it to Pier customers, because in shared servers
>>>>>> the sizes of image is very important.
>>>>>>
>>>>>
>>>>>
>>>>> The latest PharoCore 1.0 is also 7.8 MB after ScriptLoader new
>>>>> cleanUpForRelease, where Seaside and Pier can run perfectly also.
>>>>
>>>> Seems to be 10.7 MB.
>>>>
>>>
>>> Ups...sorry I mean ScriptLoader new cleanUpForProduction.    not
>>> ScriptLoader new cleanUpForRelease.
>>>
>>>
>>
>> I tried the ScripLoader new cleanUpForRelease on
>> Pharo1.0-10508-rc2dev10.01.2 and don't think
>> to work.
>>
>> I'm missing something?
>
> Yes, try cleanUpForProduction.
>
>
> Levente
>
>>
>> Cheers.
>> Germán.
>>
>
>
>
>



--
=================================================
Germán S. Arduino  <gsa @ arsol.net>   Twitter: garduino
Arduino Software & Web Hosting   http://www.arduinosoftware.com
PasswordsPro  http://www.passwordspro.com
=================================================

12