[squeak-dev] Waiting for 3.11 artifacts.

The useful part of Edgar's point was that the 3.11 folder on the ftp site sits empty.

I personally do not want to have to understand what is on the pbwiki or to navigate keith's new ways of doing things in order to play and test out a new squeak image.

What unsettles me at the moment is that two very powerful programmers are taking 3.11 in some very new directions relative to what the community is used to.

While I have a great respect in Matthew's judgment and ability to explain what he is doing, I have found from experience that Keith's notions are more of a gamble.

Keith is a gatekeeper for ideas from other communities like Ruby. He is powerfully fast. And he is focused on core changes that bring with them a lot of power and ease.

On the other hand, I am afraid I have a great trouble trying to wade into his explanations. They get (for me) too technical, too quickly. They leave me scratching my head as to what the big simple picture is all about. So I am left with no easy way of understanding or judging his work. That in itself makes me uneasy.

He and I have discussed this in private via emails.

My other concern about this release is that it is actually a development project. The code is being written concurrent with the release. Andreas made a suggestion that each release should harvest what is already available in order to reduce risk. The plan for this release (and the talents of the release team) seem to be more state-of-the-not-quite-invented-art. This is a risk to the stability and timeline of the release.

3.9 took two years and was release with so much stuff, integration bugs were almost assured. Users ran into some good ones.

3.10.2 took a year even though it was modest in scope. It too was released with some very obvious bugs that hadn't been in 3.9. The main learning that came out of it was how brittle a release became when it was dependent on MC for development.

The current 3.11 release is an effort to find a viable way to maintain a release again. But where is the project now? And where are the (understandable) artifacts?

As a bug tracker, my experience warns me of trouble when big steps are taken w/o feedback. So I am wondering if Edgar's point can't be addressed.
Could we get an alpha image of the current stable and unstable versions of 3.11 onto the ftp site?

This would serve as a check on backward compatibility and usability for 3.11. And give the rest of us something to do.

Yours in curiosity and service, --Jerome Peace


     

These are *excellent* thoughts. Perhaps we can get a bit more clarity by
summarizing the answers to the following questions (apologies if this
amounts to repeating things that have been said many times before):

1. What is the current status of 3.11 - has work on it "officially"
started? (yes/no/dunno are all fine answers here; I'm trying to
establish the basics in terms of where in the process we are)

2. What are the goals for 3.11? I have seen references to
http://installer.pbwiki.com/311 - is this "the place" for it? (again
yes/no/perhaps are all good answers, I just want to make sure we're
using the same frame of reference)

3. Where are we in the process towards these goals? Both from a
high-level perspective as well as the nitty-gritty details of things
that don't work but need to be addressed for a release.

4. How does one best track progress for 3.11? Is there an update stream?
Are there Monticello releases? Mantis entries? Installer scripts? Alpha
images? All of them?

5. How does one best contribute to 3.11? (both, for more long-term
continued development as well as the ten-minute scratch-an-itch kind of
exercise)

I think that maybe one of our problems here is that we lost a little
track of what exactly the goals for 3.11 are and where we're in the
process relative to those and I think get some clarity on that might
help for future steps.

Cheers,
   - Andreas


Jerome Peace wrote:

Jerome Peace wrote:
> The useful part of Edgar's point was that the 3.11 folder on the ftp site sits empty.
>  
Please give me a bit of a break, my mother died in March and the person
that I am a full time carer for, became elective mute for 2 months, and
has been in hospital for  3 months. I also have a day job, and haven't
got much done on the actual 3.11 in 6 months.

In actual fact I have only spent a couple of non-spare weekends on the
actual final form of the 3.11 building bit. That is the smallest part of
the 3.11 effort!

It's a bit soon to be asking for artefacts after 2 weekends work.
Perhaps 3? Having said that instructions on building 3.11 images were
published on the releases list, months ago (September I think it was),
and those scripts have been around since January last year.

There are also products out there you can look at. MC1.5, MC1.6,
Sake/Packages, Logging.

As I said before there is not huge demand for a new release of the
kernel instantly. There is huge demand for things like
LevelPlayingField, and Sake/Packages and .... there is plenty of pent up
demand for Bob the Builder which I have promised asap.
> I personally do not want to have to understand what is on the pbwiki or to navigate keith's new ways of doing things in order to play and test out a new squeak image.
>
> What unsettles me at the moment is that two very powerful programmers are taking 3.11 in some very new directions relative to what the community is used to.
>  
The alternative was officially nothing. I piped up when the board were
considering cancelling 3.11

So anything is better than nothing, perhaps?
> While I have a great respect in Matthew's judgment and ability to explain what he is doing, I have found from experience that Keith's notions are more of a gamble.
>  
So far, I am quite pleased to say that everything I have put my hand to
has worked really well.

But this comment indicates to me that you really dont "get it" yet.

The whole deal with this 3.11 project is not about delivering an image,
its about addressing a need, through putting a philosophy into practice
across the board. The 3.11 goal is to showcase the tools that make that
philosophy possible. While the tools are not ready there is no 3.11
(fortunately the tools are getting there and there will be a 3.11)

The need is definitely there, and the philosophy aiming to meet that
need has been operating well for almost a year now. That's not a gamble
at all, its already happening.

Edgar has delivered image after image, but does that help anyone in the
long run really. It  doesn't help me. I have production code and I don't
have time to spend a month moving it form one image to another manually,
without any tools to help, broken MC, broken Universes etc. It doesnt
help us move forward in the future to something like Morphic 3.0, or COG
for which atomic loading is likely to be essential.

Real World Example:

As an example, Gjallar was working in 3.8, there is no technical
compelling reason to move the huge code base over to 3.10. It doesn't
offer any must have new features. The only reason for moving is to be
able to keep up for the sake of it. So into this situation comes
Installer, Gjallar migrates to use Installer for its build scripts
(July2007). Once Installer is used, the build script can be run in 3.7,
3.8, 3.9, 3.10, Sophie, Croquet or Etoys. Installer proves the common
ground that is essential to move forward, even though that move didn't
take place for almost a year, the Gjallar team knew that they were no
longer a fork, because they had the tools to make keeping up possible
and straight forward.

So did Gjallar move to 3.10 because of the 3.10 image features, or
because Installer and LevelPlayingField helped make it a smooth
transition?  Gjallar is a fork no more. Which of 3.10/Installer is
really contributing most to moving the community forward? I would say
that Installer is doing a damn fine job for a little tool.

The  philosophy of which I speak is one of supporting everyone who is
using squeak, in every different image and fork, so that they can move
forward, hopefully on a convergent path.

Edgar still looks for the forks to agree on a common kernel, his
approach expects the other forks to follow him. What is actually
happening is more and more divergence, and if that continues squeak
3.10+ will get less and less patronage as the community spreads out more
and more.

In contrast our philosophy accepts that there are forks, and is pleased
that lots of interesting work is being done in diverse places. But we
can afford to hold off on the run of the mill process of adding 100 or
so minor fixes, preferring instead to build more common ground.

So... the 3.11 release team in recent months has worked on Installer,
Atomic Loading for MC, Sake, Sake/Packages, SUnit, Files support for MC,
Sake-Scheduler, Mantis (squeak code for using mantis data), Rio, and
Bob. All for the purposes of moving the whole community (and in
particular my paid work) forward.

Just because one possible deliverable is a 3.11 image, the image is not
the most important thing. The important thing is the amount of common
ground achieved/achievable.

Example:

Rather than maintain 10 different CollectionsTests, it would be good to
maintain just one uber-CollectionsTest package. This would be helpful in
order for the community to maintain parity with API's in all forks.

So if I am a squeak 3.10 user and I add a feature to Collections with a
test. When a 3.8 user loads the latest and greatest test cases, my new
test will break in his image, but work in mine, giving him "non green"
status.

So SUnit needs to be able to understand that it is being used in several
images, and tests need to be marked if there are any anomalies between
target image/platform/vm etc.

I wrote the code to achieve this 2 years and 3+ months ago. The code in
itself is no use, without someone putting the uberCollectionTests vision
into practice, and you dont need a 3.11 image to do it. However the 3.11
effort is aiming to showcase the collection of tools that give us this
common ground.

The alternative earlier in 2008 was that Squeak3.11 would not have gone
ahead, and my effort on improving SUnit 2 years ago would have been
wasted entirely. Thats why I took this on.

The important thing is that all of the contributions to that 3.11 are
also available to all other image users. So you don't have to wait for a
3.11 image to partake of the new wine.

Its not really a gamble its a coherent strategy to implement what Goran
had as a vision, multiple update streams, but in a different form.
Different experts can contribute different tasks managed in Monticello.
> Keith is a gatekeeper for ideas from other communities like Ruby. He is powerfully fast. And he is focused on core changes that bring with them a lot of power and ease.
>  
I dont know about that, I have been smalltalking full time for much of
14 years, rubying for about 9 months total over 8 years.

Sake was Brian's idea.
> On the other hand, I am afraid I have a great trouble trying to wade into his explanations. They get (for me) too technical, too quickly. They leave me scratching my head as to what the big simple picture is all about. So I am left with no easy way of understanding or judging his work. That in itself makes me uneasy.
>
> He and I have discussed this in private via emails.
>
>  
> My other concern about this release is that it is actually a development project.
Correct it is a development project to create tools for making releases
when the previous teams have complained about the tools.
>  The code is being written concurrent with the release. Andreas made a suggestion that each release should harvest what is already available in order to reduce risk.
Risk of what? at one point 3.11 wasnt going to happen at all, anything
more is a bonus.
>  The plan for this release (and the talents of the release team) seem to be more state-of-the-not-quite-invented-art. This is a risk to the stability and timeline of the release.
>  
Basic programming principles, three variables,
time/functionality/quality. The functionality required is essentially
fixed. The quality has to be high, but requires the test server
functionality, so time is the the variable. Its ready when its ready.

Once the tools are complete we switch to a completely different
workflow, timeboxed releases, all tests green etc etc etc.
> 3.9 took two years and was release with so much stuff, integration bugs were almost assured. Users ran into some good ones.
>
> 3.10.2 took a year even though it was modest in scope. It too was released with some very obvious bugs that hadn't been in 3.9. The main learning that came out of it was how brittle a release became when it was dependent on MC for development.
>
> The current 3.11 release is an effort to find a viable way to maintain a release again. But where is the project now? And where are the (understandable) artifacts?
The explicit 3.11 part of project was effectively delayed for 6 months
due to my personal reasons and work commitments. So please be gentle.
> As a bug tracker, my experience warns me of trouble when big steps are taken w/o feedback. So I am wondering if Edgar's point can't be addressed.
> Could we get an alpha image of the current stable and unstable versions of 3.11 onto the ftp site?
>  
Its on its way, and there have been instructions available on the
release mailing list on how to build an image yourself for several months.
> This would serve as a check on backward compatibility and usability for 3.11. And give the rest of us something to do.
>  
Have you joined the release list?
> Yours in curiosity and service, --Jerome Peace
>  

Dont Panic

Keith

Andreas Raab wrote:
> These are *excellent* thoughts. Perhaps we can get a bit more clarity
> by summarizing the answers to the following questions (apologies if
> this amounts to repeating things that have been said many times before):
>
> 1. What is the current status of 3.11 - has work on it "officially"
> started? (yes/no/dunno are all fine answers here; I'm trying to
> establish the basics in terms of where in the process we are)
I have held back from saying it is "officially" started, due to pressure
to deliver on work deadlines, which have suffered due to my extreme
domestic pressures recently. Home life has returned to normal but still
my paid work is a priority.

Nevertheless the goals were stated in the paper that Matthew presented
to the board, and those goals have been worked upon and stuff is in the
process of coming online.
> 2. What are the goals for 3.11? I have seen references to
> http://installer.pbwiki.com/311 - is this "the place" for it? (again
> yes/no/perhaps are all good answers, I just want to make sure we're
> using the same frame of reference)
"THE" place of reference is the 311-Proposal accessible from that page.

Matthew might update things in the next week or two.
> 3. Where are we in the process towards these goals? Both from a
> high-level perspective as well as the nitty-gritty details of things
> that don't work but need to be addressed for a release.
Many of the parts are in place. We are waiting for Bob to bring them all
together, Bob was waiting for Rio to support ftp seamlessly.
> 4. How does one best track progress for 3.11? Is there an update
> stream? Are there Monticello releases? Mantis entries? Installer
> scripts? Alpha images? All of them?
1. [hidden email] - receives emails of all of the
monticello package commits for the components that contribute to 3.11

2. [hidden email] - discussion on the release
(though irc is our preferred means of communication)

3. www.squeaksource.com/Tasks - has the actual build tasks, the roadmap
is embodied in this code. This is where you can contribute tasks, or
define your own personal test builds for Bob to build and test for you

4. www.squeaksource.com/Packages - loading and unloading packages, this
is where you can contribute knowledge about what works where, AND how to
unload things. e.g. "Castrait" (see previous mail) could be published
here, as could SqueakByExample. Recent addition include a Kernel-Tracer
unload, and a Null unload, a ProcessSpecific unload, AllTests (hunts
down tests for the loaded packages), and Tasks (loads tasks for this
version and the next)

5. A Mantis revival is on the way. There will be some queries for you to
see progress on bug fixing very clearly.

6. Image build script is Installer installUrl:
'http://installer.pbwiki.com/311'.  When it completes (it used to) we
will publish an image.
> 5. How does one best contribute to 3.11? (both, for more long-term
> continued development as well as the ten-minute scratch-an-itch kind
> of exercise)
a.  Small fixes and Additional Tests - ChangeSet on Mantis, AND an
Installer script on mantis.
b.  Bigger Kernel contributions add your task to ReleaseSqueak310, and
add it as a dependency, or step to one of the build steps.
c.  Kernel contributions can be made as a monticello package which is
merged into the kernel by a task.
d.  Unloading stuff - changeset on mantis, and unload script in Packages
e.  Knowledge of what loads where (basic dependencies -> Universes)
anything more Packages
f.  Tutorials/Readmes/Intrductions use the MC files feature to put the
files under monticello, and add a corresponding Packages entry for
load/unload
g. Build scripts for Bob, particularly the OneClick image
> I think that maybe one of our problems here is that we lost a little
> track of what exactly the goals for 3.11 are and where we're in the
> process relative to those and I think get some clarity on that might
> help for future steps.
hope that helps a bit

Keith

FYI: The roadmap task

regards

Keith

=======

taskTESTCANDIDATE
    ^ self define: [ :task |
           
        "task author: 'auto'."
       
        task action: {
            self taskResetFixesAutoDocumentation.
   
            self taskTidyWorld.
            self taskTidyMorphs.
            self taskTidyRepositories.
           
            self taskSetPreferences.
            self taskPackageUpgrades.
            self taskKernelImports.

            self taskBugFixes.
            self taskDocumentFixes.    

            self taskReorganizeCategories.
            self taskReorganizePackages.

            self taskCleanDeprecated.
            self taskCleanMiscellaneous.
            self taskMarkNewlyDeprecated.
           
            self taskCleanExcess.
            self taskCleanUp.
           
            self taskFinalize.            
        }.
         
    ].

taskRELEASECANDIDATE
    ^ self define: [ :task |
           
        task dependsOn: {
            PackageOrganizer default taskSaveAllPackagesIn: self repository.

            (Packages current named: 'Universes') unload.            
            (Packages current named: 'YAXO') unload.            
            (Packages current named: 'SqueakMap2 loader') unload.      
     
            (Packages current named: 'SqueakMap2 base') unload.  

            self taskRemoveTests.
            self taskFinalize.
        }.
    ].

Keith Hodges wrote:
Obviously. We all have to make a living ;-) But in the meantime we
should see if we can redistribute the work or adjust the goals to
reflect the realities we're encountering. One of the reasons why I have
proposed in the past to only include work that was completed previously
is to avoid situations where some individual needs to make adjustments
to their situation. From what you say I can only guess that there are
parts of the 3.11 proposal that critically depend on you and that you
won't have the time to work on for the foreseeable time. Would it make
sense to adjust the plan towards that end?

> Nevertheless the goals were stated in the paper that Matthew presented
> to the board, and those goals have been worked upon and stuff is in the
> process of coming online.

By "the paper" I presume you mean
http://installer.pbwiki.com/Squeak311Proposal ?

>> 2. What are the goals for 3.11? I have seen references to
>> http://installer.pbwiki.com/311 - is this "the place" for it? (again
>> yes/no/perhaps are all good answers, I just want to make sure we're
>> using the same frame of reference)
> "THE" place of reference is the 311-Proposal accessible from that page.

I have found the following three pages:
   http://installer.pbwiki.com/311
   http://installer.pbwiki.com/Squeak311
   http://installer.pbwiki.com/Squeak311Proposal
Is this it or are there other places that I'm missing?

> Matthew might update things in the next week or two.
>> 3. Where are we in the process towards these goals? Both from a
>> high-level perspective as well as the nitty-gritty details of things
>> that don't work but need to be addressed for a release.
> Many of the parts are in place. We are waiting for Bob to bring them all
> together, Bob was waiting for Rio to support ftp seamlessly.

Could you be a little more precise about which parts are in place? The
above three pages list lots of stuff and it is very difficult to
understand how much progress has been made where, what dependencies
remain and where (on a percentage or similar base) we are in the process.

>> 4. How does one best track progress for 3.11? Is there an update
>> stream? Are there Monticello releases? Mantis entries? Installer
>> scripts? Alpha images? All of them?
> 1. [hidden email] - receives emails of all of the
> monticello package commits for the components that contribute to 3.11

I've briefly looked at it and I don't understand the purpose. There seem
to be hundreds of commit messages about Monticello, Installer, Packages,
Sake, Tasks etc. And zero about Kernel, System, Graphics etc. Which
doesn't look like a 3.11 commit list but rather like a KPH-dev list ;-)
(nothing wrong with that but in terms of tracking 3.11 progress I would
expect to see just as much traffic in other areas)

> 2. [hidden email] - discussion on the release
> (though irc is our preferred means of communication)

If you don't mind, can we have these discussions on Squeak-dev? I never
liked the idea of "private" release lists too much - I think the release
is the most important artifact produced here so why not discuss it out
in the open so that people can follow along?

> 3. www.squeaksource.com/Tasks - has the actual build tasks, the roadmap
> is embodied in this code. This is where you can contribute tasks, or
> define your own personal test builds for Bob to build and test for you

How does one read this? When I look at the code there is a
taskRELEASECANDIDATE which (from what I can tell) unloads a bunch of
packages, unloads the tests and that's it. No fixes, no loading of any
of the new stuff you were referring to above? Or am I misreading this?

> 4. www.squeaksource.com/Packages - loading and unloading packages, this
> is where you can contribute knowledge about what works where, AND how to
> unload things. e.g. "Castrait" (see previous mail) could be published
> here, as could SqueakByExample. Recent addition include a Kernel-Tracer
> unload, and a Null unload, a ProcessSpecific unload, AllTests (hunts
> down tests for the loaded packages), and Tasks (loads tasks for this
> version and the next)

Again, I'm not sure what exactly I'm seeing here.

> 5. A Mantis revival is on the way. There will be some queries for you to
> see progress on bug fixing very clearly.

That would be very useful.

> 6. Image build script is Installer installUrl:
> 'http://installer.pbwiki.com/311'.  When it completes (it used to) we
> will publish an image.

Thanks, I'm running it now.

It sure does.

Cheers,
   - Andreas

Andreas Raab wrote:
>> 6. Image build script is Installer installUrl:
>> 'http://installer.pbwiki.com/311'.  When it completes (it used to) we
>> will publish an image.
>
> Thanks, I'm running it now.

This blows up when InstallerMantis tries to load a gzipped fix. I think
the fix is this:

InstallerMantis>>installDefault: aFileName from: stream
        (aFileName endsWith: '.gz')
                ifTrue:[stream contents unzipped fileIn]
                ifFalse:[stream fileIn].

But I couldn't really test this since I wasn't able to resume the
process where it died.

Cheers,
   - Andreas

Hi Andreas,

Thanks for having a go!
> Obviously. We all have to make a living ;-) But in the meantime we
> should see if we can redistribute the work or adjust the goals to
> reflect the realities
volunteers? yes Thats it.
   1. Start with 3.10 LPF - [ works well ]
   2. Sake and Tasks - [ loading is now in Packages item 5 ]
   3. Do SetPreferences-Squeak3:10  [ is a roadmap task ] [background
color is set for image type tc/rc/u etc ]
   4. Add 311-KernelExtensions - as version managed Sake/Task [ Being
broken down into mantis fixes ]
   5. Packages [ working well - Matthew thinks we need a UI ]
   6. Do Reorganize-Squeak3:10 [ in progress ]

[ Manual Fixes - we have 80 or so in the system, as you pointed out
Installer is struggling with .gz extension
  but that is a trivial thing to fix ]

[ Plan to automate more of the fixing process - We have "Mantis" code
which reads Mantis periodically for Bob
  Mantis will generate tasks for applying fixes according to status.
(1/2 day) ]

[ Mantis review - 1 week initially - then how long is a piece of string
- (850 unresolved issues to go) ]

   7. Do Latest
         1. MinorFixes-Squeak3:10 - as version managed SakeTask
         2. MajorFixes-Squeak3:10

         3. PackageUpgrades [ task present in road map - complete? ]

[ 8 Same as 7 only Unstable ]

   8. Optionally Do LatestUnstable
         1. MinorFixesUnstable-Squeak3:10
         2. MajorFixesUnstable-Squeak3:10
         3. PackageUpgradedUnstable

   9. DeprecatedMark-Squeak3:10 - mark deprecated
  10. DeprecatedClean-Squeak3:10 - remove stuff that was deprecated in
3.9/3.10 [done]
  11. Save Packages [done]
  12. Clean-Squeak3:10 - remove stuff that is old or can be reloaded [
Some done - sufficient to be an example ]
  13. Strip-Squeak3:10 - back to minimal [ Some done - sufficient to be
an example ]

>>> 4. How does one best track progress for 3.11? Is there an update
>>> stream? Are there Monticello releases? Mantis entries? Installer
>>> scripts? Alpha images? All of them?
>> 1. [hidden email] - receives emails of all of the
>> monticello package commits for the components that contribute to 3.11
>
> I've briefly looked at it and I don't understand the purpose. There
> seem to be hundreds of commit messages about Monticello, Installer,
> Packages, Sake, Tasks etc.
Given that our goal is to put in place a process for generating images
with Bob. Take for example the step of loading a workspace with a ReadMe
and an Introduction Release Notes etc. These steps have been done
manually for years, and there is no Monticello cabability for
collaboratively managing text files, and Workspace is hopeless at
saving/loading files

So...

The Commit Messages you see are:

Monticello - Adding files support.
Installer - This is a brand new Installer implementation with the same
API (thanks to Matthew)
Tasks - thats what is specifying the build so adding a Packages beta
unload: 'Kernel-Tracer'. to a task is matched by.....
Packages - adding a PackagesBeta-#KernelTracer entry
> And zero about Kernel, System, Graphics etc. Which doesn't look like a
> 3.11 commit list but rather like a KPH-dev list ;-) (nothing wrong
> with that but in terms of tracking 3.11 progress I would expect to see
> just as much traffic in other areas)
The fixes applied are not commited until the first step of the
taskRELEASECANDIDATE (see previous road map  email) and we havent been
there for a while.
>> 2. [hidden email] - discussion on the release
>> (though irc is our preferred means of communication)
> If you don't mind, can we have these discussions on Squeak-dev? I
> never liked the idea of "private" release lists too much - I think the
> release is the most important artifact produced here so why not
> discuss it out in the open so that people can follow along?
I do mind actually for various reasons (private mail)
>> 3. www.squeaksource.com/Tasks - has the actual build tasks, the roadmap
>> is embodied in this code. This is where you can contribute tasks, or
>> define your own personal test builds for Bob to build and test for you
> How does one read this? When I look at the code there is a
> taskRELEASECANDIDATE which (from what I can tell) unloads a bunch of
> packages, unloads the tests and that's it. No fixes, no loading of any
> of the new stuff you were referring to above? Or am I misreading this?
taskTESTCANDIDATE is run first this generates a version of 3.10 with
stuff added

taskRELEASECANDIDATE starts with the output of taskTESTCANDIDATE and
just tidies it up and updates the version number.
>> 4. www.squeaksource.com/Packages - loading and unloading packages, this
>> is where you can contribute knowledge about what works where, AND how to
>> unload things. e.g. "Castrait" (see previous mail) could be published
>> here, as could SqueakByExample. Recent addition include a Kernel-Tracer
>> unload, and a Null unload, a ProcessSpecific unload, AllTests (hunts
>> down tests for the loaded packages), and Tasks (loads tasks for this
>> version and the next)
> Again, I'm not sure what exactly I'm seeing here.
Most of the load/unload scripts are handled by the #defaultAction which
simply uses the data supplied in

self info url: 'http://host/project/package-author-1.mcz'.

Any lump of code that can be moved into a package is committed to
squeaksource.com/311 and gets an entry in Packages, probably in
PackagesDev, or PackagesBeta, so that it can be reloaded. If we would
rather keep that lump in the release, it gets an entry in Packages so
that it can be unloaded easily later. http://installer.pbwiki.com/Squeak311Proposal

The only goal left in that page to be completed is Bob, and underscores
in selectors.

The goals for 3.11 are fundamentally based around getting Bob working,
and to showcase a release being built with Bob.
The more work is done on the image stuff the less work is done on Bob.

[ Bob status - Bob is probably less than 1 perfect working day away from
release ].
>>> process relative to those and I think get some clarity on that might
>>> help for future steps.
>> hope that helps a bit
>
> It sure does.
>
> Cheers,
>   - Andreas
thanks for having a go.

Keith

Hi all!

First of all - great thread!! :) I could ask questions and comment on
hundreds of things but I am letting the current people hash it out for
now. One thing I would like someone to do though is to "distill" this
thread into at least a Swiki page on the Squeak Swiki! :)

But a bit of comments anwyay:

Keith Hodges wrote:
> Jerome Peace wrote:
>> I personally do not want to have to understand what is on the pbwiki or to navigate keith's new ways of doing things in order to play and test out a new squeak image.
>>
>> What unsettles me at the moment is that two very powerful programmers are taking 3.11 in some very new directions relative to what the community is used to.
>>  
> The alternative was officially nothing. I piped up when the board were
> considering cancelling 3.11
>
> So anything is better than nothing, perhaps?

Indeed and I have repeatedly asked for a clarification on the 3.11
status and how "official" it is! Now I hope it is FULLY official and
that Matthew (?) is the release team leader.

Many of us who have been around for a while know that we have tried
numerous approaches over the years and we know that many of these have
failed due to various reasons. For example, the idea of "harvesting" and
having so called "harvesters" failed because it only led to a few people
burning out (in the very early years it "worked" because we had paid
people doing the harvesting).

So such a model is not viable IMHO. We need to move and try other routes
- and I for one applaud the current effort in 3.11 and will try to chip
in wherever I can.

Yes, we made the move just recently of Gjallar over to 3.10 and adjusted
our Installer script to use LPF etc. It worked really well and we did it
mainly because we don't want to be left out on the improvements/fixes
pouring in. All in all Installer is a great tool making "image building"
quite easy. Combined with Sake/Packages (modulo having not used it yet)
I presume it gets even more powerful.

A trivial example from Gjallar: We include fixes available on Mantix
using Installer "oneliners". We don't need to wait for someone else to
harvest it, get it into the image etc etc.

[SNIP]
> The important thing is that all of the contributions to that 3.11 are
> also available to all other image users. So you don't have to wait for a
> 3.11 image to partake of the new wine.
>
> Its not really a gamble its a coherent strategy to implement what Goran
> had as a vision, multiple update streams, but in a different form.
> Different experts can contribute different tasks managed in Monticello.

Yes, we share the same understanding of where we are right now - the
Squeak world is already "forked" in several directions. We need to get
mechanisms in place to cope with that. I still hope to be able to move
DeltaStreams forward (time, time, time...) but the important thing is
that we are several who share the understanding of the core problem.

> Dont Panic

Hehe, I like it! :) Remember folks - we are all in this for fun! Matthew
and Keith (and several others of course) have made huge contributions
over the last year, and I am extremely grateful for that.

I hope to be able to pull my share mainly related to SM.

regards, Göran

>>> 2. [hidden email] - discussion on the release
>>> (though irc is our preferred means of communication)
>> If you don't mind, can we have these discussions on Squeak-dev? I
>> never liked the idea of "private" release lists too much - I think the
>> release is the most important artifact produced here so why not
>> discuss it out in the open so that people can follow along?
> I do mind actually for various reasons (private mail)

?

makes me very curious. could I have a copy of that private mail :)


Stef

"Keith Hodges" <[hidden email]> wrote in message

> ...

Thank you for all the work. Even just Installer has been a very big help to
me.

> Installer,
> Atomic Loading for MC, Sake, Sake/Packages, SUnit, Files support for MC,
> Sake-Scheduler, Mantis (squeak code for using mantis data), Rio, and
> Bob.

Only if this has an easy answer ... which one(s) of these correspond most
directly to ruby-gems: dependency and version-aware package build + install
+ remove, based on a package repository ?

Sophie

Keith Hodges wrote:
> Jerome Peace wrote:
>  
>> The useful part of Edgar's point was that the 3.11 folder on the ftp site sits empty.
>>  
>>    
> Please give me a bit of a break, my mother died in March and the person
> that I am a full time carer for, became elective mute for 2 months, and
> has been in hospital for  3 months. I also have a day job, and haven't
> got much done on the actual 3.11 in 6 months.
I'm really sorry to hear this.
Take care of your self because there is a very high burnout rate of
release coordinators!!
Squeak should be fun, not worth risking personal health over.

Karl

Keith Hodges wrote:

disclaimer - i am just a user dabbling in Squeak a bit. Feel free to
ignore me.

*snip*

So, basically these are the two philosophies:

Installer --> to be able to load code in many versions/forks

Images --> predefined load

Is that about correct?

Assuming yes:
To me, images appear to be a consequence of a working Installer/Packager.

For example, take VAST/VS(E) (even VW):
You develop your code in some image. Then you create a package for your
code. The code still runs in your dev image.
However, the code also runs - assuming Envy does not do a mistake
gathering all dependencies (VAST) /you do not mess up the SLL bindings
(VS(E) -  once loaded in a plain image of around 50k.

So, there you go: Create a package, load it in some plain image, then
you have a (insertFeatureHere:aFeature)-Image.

Is "core" Squeak supposed to be that different? What is the vision, what
is the main branch?
Why is there such a hassle about this topic?

(this does not even take into account interfacing to other Smalltalk
dialects)

*snip rest*

A Gjallar is a good example of doing things right.
It doesn't need to stick with XXX image to the end of days, instead -
Gjallar evolves along with tools/frameworks it using.
This means that it using quite agile/modular development strategy:
 it can always migrate to a new version of seaside, or magma, or image.
Also, it makes easy to try different combinations or new releases of
components it relies on, which in own turn generating a valuable
feedback for them, and generally helps making better, faster, bug-free
frameworks which Gjallar is using.

--
Best regards,
Igor Stasenko AKA sig.

Claus Kick wrote:
> Why is there such a hassle about this topic?

Good question. It seems keithy's Installer would be darn useful for
edgar's image building.

Hi Göran -

I'm a big fan of builds in production systems but I'm a little at a loss
how exactly you are using Installer and how it helps you address
particular needs. At Qwaq, we use Monticello exclusively and the way
we're working is that we have our own versions of *all* the Monticello
packages we depend on that we modify when we have to. What it means is
that we have a consistent development process, don't rely on any
external servers and that (for example) integrating a fix is as simple
as loading it and committing to the repository which is consistent with
the rest of the development activities. We can do this both in our own
code as well as in the base code and we don't need to wait for anyone
before the next product build can go out.

In Gjallar, how do you deal with the same set of problems (making a
modification for the product, integrating some external modification)
using Installer? In particular how do you deal with issues like having
to rely on external servers (which might be on the other end of the
world, down and out etc), the ability to make some changes quickly for
the product to ship etc.

I have never found a production setup which did not require to
ultimately have your own copies of the code on your own servers in which
case you're quickly getting to the point that a single homogeneous
environment is advantageous (like using MC+configs, or using an update
stream) and where a mix of these ultimately only gets you into trouble.
The reason why I like DeltaStreams is that it sounds like an approach
that allows one to reconcile the (necessary) local mods with other
changes more easily than could be done by MC or updates. But I don't
think Installer is quite that and so I'm not sure how it helps in the
process. An experience report would be most welcome.

The other question here is that I don't understand how Installer has
helped you moving between versions. Having moved Croquet from 3.2 to 3.4
to 3.6 to 3.8 I have found each version a completely unique challenge
and I cannot imagine that any particular tool would have helped me
getting this work done more efficiently than the (extremely painful)
process of loading, having it break, figure out where exactly that
interface went, rinse and repeat. Anything that could help with this
process would be hugely beneficial.

Cheers,
   - Andreas

Göran Krampe wrote:

Hi Keith -

Keith Hodges wrote:
> Edgar has delivered image after image, but does that help anyone in the
> long run really. It  doesn't help me. I have production code and I don't
> have time to spend a month moving it form one image to another manually,
> without any tools to help, broken MC, broken Universes etc. It doesnt
> help us move forward in the future to something like Morphic 3.0, or COG
> for which atomic loading is likely to be essential.

There is only one way to answer your question: YES!!! It *does* help to
produce alpha images.

What you have to distinguish here is the need of you personally and the
need of the community. For this community, an image is a form of
communication that it is familiar with. For this community, the
announcement of the availability of a new alpha/beta/gamma image is a
sign of life, is a documentation of the process that the release team
goes through.

3.11 has lost *all* visibility to me. I didn't know whether it had even
started because there was nothing on the server, there were no
announcements on this list, so how would people know? I didn't know how
to check in on the progress because 3.11 used Installer scripts -
something that this community at this point in time is simply not
familiar with.

Once I found out what to run I spent about half an hour to load all the
code. At the end of the process Installer blew up on me. Again, having
an image would have addressed this problem.

So there is *lots* of good reasons to produce alpha images. Hey, and
given that Installer is automated this shouldn't be a big deal right?
And someone other than you should be able to do it too, right?

*Please* do realize the value of alpha images as a form of communication
in this community. If you post a fix for the .gz loading I will happily
produce an alpha image for people to play with.

Cheers,
   - Andreas

Hi Keith,

On Tue, Dec 09, 2008 at 01:00:36PM +0100, G?ran Krampe wrote:
>
> Many of us who have been around for a while know that we have tried
> numerous approaches over the years and we know that many of these have
> failed due to various reasons. For example, the idea of "harvesting" and
> having so called "harvesters" failed because it only led to a few people
> burning out (in the very early years it "worked" because we had paid
> people doing the harvesting).

I cannot help but suspect that the task of harvesting is inherently
hard work that requires human communication, intelligence, and
motivation. While the work certainly can be streamlined with
better tools, it should also be possible to improve by making
it more enjoyable.

That's the reason that I like Edgar's emphasis on "Fun Squeak",
regardless of the specific tools employed. It is also the reason
that I liked the old BFAV (Bug Fix Archive Viewer) process. The
tool had problems, but using it and participating in the process
was enjoyable and accessible to the entire community.

The harvesters approach did not fail.The quality of the work
was very good, and the results were both successful and broadly
accepted by the community. That sounds like success to me.

Dave

Andreas Raab wrote: I didn't say there was anything wrong with alpha images. What I said was
that an alpha image doesn't help me with the integration process, if I
have legacy code.

If I change an API in 3.11, and I have a package version I am developing
in 3.11, I will break all the code that relies on the new API, and my
package will not work for those on older images. Clients of my package
will have to stick with the old version of my package until they get to
the image transition point, and then they will have to port to my new
version.

However if I have a place in the framework to publish the new api for
older images, then this allows my package to continue to be supported in
older images even though it is using the upto date api.

The clients can keep up to date with my changes to my package, and when
it comes to transitioning to a new image, there is nothing required for
it to work.

This is what LevelPlayingField does. So I am simply proposing that an
alpha image without thought for a compatability layer is r&d.

Keith