[squeak-dev] 3.11 and the trunk

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
43 messages Options
123
Reply | Threaded
Open this post in threaded view
|

[squeak-dev] 3.11 and the trunk

Andreas.Raab
Folks -

This is going to be lengthy, so I apologize beforehand. But we're in an
unfortunate situation right now, an impasse between the board-approved
3.11 proposal and the new kid in town, the trunk. I'm trying to get out
of this impasse, so here are some thoughts.

First of all, I made a mistake. After my discussions with Matthew (both
on- and off-list) I fully expected that 3.11 was dead when Matthew told
me that he would stop working on it. Keith had been pretty much
invisible in the discussions up to that point so I had (incorrectly)
assumed that he wouldn't have any attachment to the 3.11 label and
consequently I didn't spend any time on thinking about the relationship
between 3.11 and the trunk. Obviously that was wrong. So apologies again
for misinterpreting the situation.

That out of the way, I have to admit that I only realized today just how
different the goals of the 3.11 proposal were from what I have been
reading into them the entire time.

Quoting http://installer.pbworks.com/Squeak311Proposal:
        "The technical goal of Squeak 3.11 development is to provide a tool
chain to enable many contributors to add code to the base without
bottlenecks. The validation of this goal being the production of a new
3.11 image built using this tool chain."

What I had missed (literally until today) is that the goal of this
proposal is not to create a new version of "Squeak the software", the
goal of the proposal is to develop a set of tools. The creation of a
3.11 release artifact in this proposal is just a validation step and not
given much further thought (just check how detailed the description of
the tools is and how vague the description of the release artifact itself).

I never got this before. And I'm sure many others didn't. And I'm not
even sure the former board fully realized that they were ordering a
kitchen instead of the entrée.

But suddenly things make sense that previously didn't. It does explain
for example the contents of http://ftp.squeak.org/3.11 (which had
puzzled me endlessly). Since the goal of the proposal wasn't to create a
3.11 image we find nothing that's labeled 3.11 in the directory. Instead
we find different Squeak versions that the tools were applied to. Makes
sense once you understand that the proposal isn't trying to create a new
release of Squeak but rather a set of tools.

The trunk development on the other hand is all about Squeak the
software, the content. I don't care about tools any more than that I
need them to get the job done, and if I can beat Monticello into
submission and have it give me an incremental update stream with minimal
fuzz I will do that and concentrate on what I consider the important
part: Squeak.

In order to get out of our current impasse, I first and foremost need
people to understand this: The trunk is not about tools or
infrastructure and the 3.11 proposal is not about content.

And perhaps that is also showing the way out. What if we left the tools
work to the tools and the development work to the developers? What I
mean is that we can use the tools that have been developed in the 3.11
process, for example, to do continuous integration from Mantis. All the
bits are there (well, were there, now that Bob got pulled, so perhaps
this won't work) but the tools infrastructure can certainly be used to
take part in the content development which happens in the trunk.

The result of which would be two parts: A loadable set of tools,
developed in the 3.11 process, just as stated in the 3.11 proposal,
applicable to different Squeak versions, just as illustrated by the
content of the 3.11 directory. But in addition to the tools we also get
a rich Squeak 3.11 release artifact the result of work that the tools do
using Mantis, and the work that the developers do using the trunk.

In practice, this would mean we should start the integration process
sooner rather than later, and I'm willing to spend a bit of time in
setting up a server, which, using the tools developed already, find new
stuff on Mantis, load it, test it, provide feedback and recommendations.
Possibly even push it straight into the inbox. The fixes already
screened and marked pending should go into into the trunk ASAP.

Does this make sense? I think it's really important not to miss the
difference in goals between the 3.11 proposal (developing tools for
Squeak) and the trunk (developing Squeak itself). And I think we can do
both at the same time without being in an endless heated conflict.

Comments welcome.

Cheers,
   - Andreas

Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] 3.11 and the trunk

Michael Haupt-3
Hi Andreas,

what an interesting turn.

Over the past few weeks, I've been wondering what actually was the
problem, and why (and how) things could possibly have gotten so heated
(the usual OSS community factors aside). My suspicion was that there
must have been some gross misunderstanding, coupled with a lack of
proper communication due to hurt feelings and refusal to respond to
rather personal attacks. I'm sorry to see I seem to have at least
gotten the misunderstanding thing right.

I don't want to start new trouble, but I have an urgent question now.
The decision that has, in part, caused the turmoil was a board
decision, if I recall that correctly. If so, how could the board as a
whole have misinterpreted the 3.11 efforts - was there not enough
discussion?

The past aside, although it is not really up to me to make peace, what
about the news ...

On Wed, Aug 19, 2009 at 8:12 AM, Andreas Raab<[hidden email]> wrote:
> In order to get out of our current impasse, I first and foremost need people
> to understand this: The trunk is not about tools or infrastructure and the
> 3.11 proposal is not about content.

Regarding the trunk, I have had the impression that the entire work
was indeed *just* about software. I may have missed some things, but I
had not quite understood what would eventually constitute release
3.11. That said, I'm a newbie in the development crowd, and this can
well be blamed on my lack of experience.

Having learned about the Installer (and related tools), I was
nevertheless wondering how - in spite of their undoubtable merits -
such tools could possibly have been excluded from usage in ongoing
development. Tools like these are clearly missing.

> And perhaps that is also showing the way out. What if we left the tools work
> to the tools and the development work to the developers? What I mean is that
> we can use the tools that have been developed in the 3.11 process, for
> example, to do continuous integration from Mantis. All the bits are there
> (well, were there, now that Bob got pulled, so perhaps this won't work) but
> the tools infrastructure can certainly be used to take part in the content
> development which happens in the trunk.

I, for one, like this idea. The tools are useful for more than just
3.11, right? But still, "getting there" would, I imagine, be easier
*with* them, rather than without.

Is there any chance of getting the entire set back? Or is there too much grudge?

Come on, this is about Squeak, whose progress most of us should be
interested in. ;-)

Best,

Michael

Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] 3.11 and the trunk

Göran Krampe
In reply to this post by Andreas.Raab
Hi!

Andreas Raab wrote:
>  And I think we can do
> both at the same time without being in an endless heated conflict.
>
> Comments welcome.

I admit (and I think most of us do) that I haven't fully read all posts
in all this and thus I really don't grok all angles BUT:

- I must say I am surprised to hear that the board, including you
Andreas, have totally "missed" that 3.11 was about tools & process and
not about an artifact. I mean, it must have been said about 137 times!
One can argue about if it was a smart move to designate a release to the
task of finding a process and getting tools in shape - but I find it odd
that this comes as a revelation this late in the game.

- Keith has more or less said that the trunk effort is counter
productive in comparison to his/their 3.11 effort. Personally I might
think it is a gray area, but even so - perhaps once and for all - with a
pedagogic mindset - explaining to us all how this is so and also
describing "trunk" and "bob" in some way that all of us can grok it...
would let us come to some conclusion.


Finally:

At this point I don't really give a rats about the above! :) I just want
us to heal, embrace, drink beer and find a way to move on. This "war"
has hurt us I think, I always brag about Squeak/Smalltalk being a very
friendly place, but it has slipped a bit lately. And thinking that it
might have been my posts setting off the fire doesn't really make it
feel better ;)

BTW, can we please get our current status/process "trunk" or "bob" or
whatever we are using these days written down on squeak.org in an
official manner? Perhaps this is already done.

This also goes for the 3.11 team - does it even exist anymore? Let us
get our act together.

regards, Göran


Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] 3.11 and the trunk

Ian Trudel-2
In reply to this post by Andreas.Raab
Hello Andreas,

I have always thought from the beginning that both could be integrated
with each other, the process and the trunk. They are not mutually
exclusive. There are however few problems:

   * The 3.11 proposal is unclear if it takes people weeks or months
to understand important parts of it.
   * Keith made it feel as such the 3.11 proposal and the trunk could
never live along side.
   * Bob is closed source and unlikely to be otherwise.

It's certainly possible to create an infrastructure based on ideas
developed in 3.11 proposal and using existing tools applied on current
trunk and/or other means. This could further be supported by a small
unpretentious Release Builder as a replacement to Bob, something that
would not have to harvest into Mantis and alike (considering the
amount of work put into Bob, it should probably focus on simplicity).

Regards,
Ian.
--
http://mecenia.blogspot.com/

Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] 3.11 and the trunk

Igor Stasenko
It is clear that for different people, who get used to one or another tool,
sometimes, it is very hard to convince them to change their habits and
stop using old tool and start using a new one
immediately.
This is lenghtly process which could take years , and we should be patient.

I am understanding the 3.11 effort first, as a set of tools which has
to be written in order to simplify/automate
an image building process and also enable a more open, friendly
contribution process with less bottlenecks, with easily crossing fork
'barriers'.
So, i had never underestimated the value of such tools. But as i said,
it could take years before people take a full grasp of the idea. So,
patience, patience and patience.

Both 'trunk' and '3.11' depicting some kind of process.
And as Andreas stated, they are perpendicular in respect that they
having different targets/goals.
It may be hard to see, because both are about software development..
And i must say that 3.11 cares more about integration of already
existing artifact(s), not really matter what they are, while trunk
process is cares more about continuous development of a single
artifact (a Squeak release).

I am proposed already, to look for a ways, how Sake could help us in
developing a squeak release. I thought that it will be a first, little
baby-steps of all devs who involved currently in trunk to get
accustomed with new tool(s) , provided by Keith for us.
At first, i propose to write down a task for building/updating the
image from trunk , as an alternative to MCMUpdater.
I am really interesting in what places we could easily replace one
tool by another, but it is not really necessary to replace it, there
is _absolutely_ could be a cases where we could use both of them. Sake
is 'Make', which could use __any existing tool__ in order to build a
final artifact.
By writting down a set of automated, test-driven and repeatable tasks,
we will entering another dimension - where we could have a fully
automated build farm which doing nightly boring job for us. Hence the
Bob.

I seen a very good progress last couple of months around Squeak, since
we established a trunk process. And i am sure, more fun is waiting for
us over a next months, when we start using a new tools, such as
Bob/Sake, DeltaStreams and MC 1.6 / MC 2.0.


--
Best regards,
Igor Stasenko AKA sig.

Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] 3.11 and the trunk

Randal L. Schwartz
In reply to this post by Michael Haupt-3
>>>>> "Michael" == Michael Haupt <[hidden email]> writes:

Michael> I don't want to start new trouble, but I have an urgent question now.
Michael> The decision that has, in part, caused the turmoil was a board
Michael> decision, if I recall that correctly. If so, how could the board as a
Michael> whole have misinterpreted the 3.11 efforts - was there not enough
Michael> discussion?

[Donning my board member hat...]

To put it simply, it started with a misunderstanding on my part.

I had seen all of the tools being generated by Keith as "means to an end"... a
new better process for generating 3.11 or 4.0 or whatever else we were aiming
for.  What I had missed, despite repeated attempts by Keith to tell me (as I
can now see in retrospect), was that an *actual* new *official* image wasn't
really his personal goal. I think I was confused by it having been called
3.11: I was expecting a 3.11 release "some day".

So, when it came up in the board meeting recently that while we were waiting
for 4.0 to get done (the licensing release), we were losing a way to get
people contributing, and had been for quite some time.  4.0 was getting
delayed week-by-week, and there hadn't been a new "core" release in over a
year.  The success of the Pharo fork proved to us that the core of squeak was
stalled.  So it was time for a reboot... get people back on.

When Andreas proposed the new trunk using existing tools, it looked like a way
to at least capture the interim desire-to-contribute, realizing that any
patches applied to 3.10.2 would have to be reharvested for 4.1 after 4.0 was
generated.  But at least we had something we could bring to squeak-dev.

There was *never* any intent to discount what Keith had accomplished. But at
the end of the day, what mattered was people in squeak-dev being told how to
help, and having a way to help, and that wasn't happening. Part of this was
because the tools weren't ready for prime time yet...Keith told me a few days
ago that he still needs another month or two.  Part of this was because of
something I wasn't aware of until recently... Keith hadn't been permitted to
speak on squeak-dev directly because of some work issues.

There's a lot of monday-morning quarterbacking here.  We could have spent more
time to understand Keith's tools and goals.  We could have made it clear that
work on the trunk *will* need to be reharvested when we get to the 4.0 code
base, either using the same tools again, or using Keith's tools if they're
ready.

But the problem we were solving was the loss of interest in helping the squeak
core move forward.  And in that respect, despite some of the negative fallout,
I believe our overall goal has been met, and supported by recent activity.

Going forward, we (as a community) need to figure out if we're generating a
3.11 release from the trunk, or whether this trunk is now just a testbed for
good ideas for the 4.1 release.  Part of this depends on the 4.0 release
timing, and that depends on the release team.  I'm working with Matthew now to
figure out what resources are needed for that.

I'd also like to get this work back into the hands of the teams.  The board
stepped in to solve the problem that didn't appear to be solving itself.  But
ultimately, it's the release team(s) who should be driving this.

--
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
<[hidden email]> <URL:http://www.stonehenge.com/merlyn/>
Smalltalk/Perl/Unix consulting, Technical writing, Comedy, etc. etc.
See http://methodsandmessages.vox.com/ for Smalltalk and Seaside discussion

Reply | Threaded
Open this post in threaded view
|

[squeak-dev] Re: 3.11 and the trunk

Andreas.Raab
In reply to this post by Göran Krampe
Göran Krampe wrote:
> - I must say I am surprised to hear that the board, including you
> Andreas, have totally "missed" that 3.11 was about tools & process and
> not about an artifact. I mean, it must have been said about 137 times!
> One can argue about if it was a smart move to designate a release to the
> task of finding a process and getting tools in shape - but I find it odd
> that this comes as a revelation this late in the game.

I can't speak for the board, only for myself. From my perspective this
is an issue of mislabeling. When I see something called "3.11" I expect
this to be about Squeak 3.11, the software, not a set of tools. I'm also
a little at a loss why then, if this is all about tools, so many noises
are made about the 3.11 release artifact. If the entire goal of the
process is to be able to load these tools into arbitrary versions, then
why don't we just load them into the trunk and get over it? There is
clearly still a level of disconnect somewhere but I'm hoping it's not
quite as drastic as the last one.

> - Keith has more or less said that the trunk effort is counter
> productive in comparison to his/their 3.11 effort. Personally I might
> think it is a gray area, but even so - perhaps once and for all - with a
> pedagogic mindset - explaining to us all how this is so and also
> describing "trunk" and "bob" in some way that all of us can grok it...
> would let us come to some conclusion.

It's quite simple really. Software development works by developers
working together in a shared repository, right? That's the trunk. The
trunk is nothing but a shared repository that the developers have commit
rights to. In the course of software development, builds happen on a
regular basis. That's Bob. Bob is a tool to assemble various bits of
code and content and compile this into a result. Simple as that.

Mantis role in this process with regards to code is to assemble
contributions from non-core-developers in the project; people who are
not given direct commit rights to the repository. These contributions
need to be reviewed and integrated. That's Installer for you. The result
of the integration process goes back into the repository. There's the
trunk again. All the parts have their place in the process.

> At this point I don't really give a rats about the above! :) I just want
> us to heal, embrace, drink beer and find a way to move on. This "war"
> has hurt us I think, I always brag about Squeak/Smalltalk being a very
> friendly place, but it has slipped a bit lately. And thinking that it
> might have been my posts setting off the fire doesn't really make it
> feel better ;)

I'm not unhappy about it. We need to get moving again. Whatever triggers
this process is a good thing. And I never expected the process to be
painfree, but I do think we are making progress if only by illustrating
that other models of contributions can actually be successful. Now we
have to work out how to make this work together to produce an actual
piece of software called Squeak.

> BTW, can we please get our current status/process "trunk" or "bob" or
> whatever we are using these days written down on squeak.org in an
> official manner? Perhaps this is already done.

I have nothing on Bob but I think the most relevant links are these:

http://installer.pbworks.com/Squeak311Proposal
http://board.squeak.org/2009/07/02/a-new-community-development-model/

> This also goes for the 3.11 team - does it even exist anymore?  
> Let us get our act together.

I leave it to Matthew to respond to this question.

Cheers,
   - Andreas

Reply | Threaded
Open this post in threaded view
|

[squeak-dev] Re: 3.11 and the trunk

Simon Michael
In reply to this post by Andreas.Raab
This seems to clarify quite a bit, and the proposal seems a good and natural next step. Also I think the board has been
doing an excellent job, both in moving things forward and in their handling of serious flamage from Keith.


Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Re: 3.11 and the trunk

Eliot Miranda-2


On Wed, Aug 19, 2009 at 9:10 AM, Simon Michael <[hidden email]> wrote:
This seems to clarify quite a bit, and the proposal seems a good and natural next step. Also I think the board has been doing an excellent job, both in moving things forward and in their handling of serious flamage from Keith.

Can we please refrain from inflaming the situation further.  If one reads Keith's postings in their entirety they seem extremely patient and discursive.  Accusing Keith (unjustly IMO) of flamage isn't going to help us resolve these issues and get on with things.

best
Eliot



Reply | Threaded
Open this post in threaded view
|

[squeak-dev] Re: 3.11 and the trunk

Simon Michael
I have been reading and have a different impression, and I wanted that on record. Perhaps I should have stuck to
expressing support. I apologise to all for adding heat.


Reply | Threaded
Open this post in threaded view
|

[squeak-dev] Re: 3.11 and the trunk

Jecel Assumpcao Jr
In reply to this post by Andreas.Raab
I did not have any problems understanding that Keith had proposed what
Andreas has just described. But when voting for the current process, I
asked if it was going to be compatible with the ongoing 3.11 effort and
was told yes. Even though I now see this was due to some confusion, I
still believe "yes" was actually correct and is what is now being
detailed.

To me, it isn't at all Keith's fault that people didn't understand what
he was trying to do. Nor do I blame anyone who didn't get it before now
(or still doesn't understand). The problem is mostly with the various
communication channels we have. There are several examples I could give
of failing miserably to explain something over email (even with attached
drawings) to really brilliant people while at the same time being able
to explain the exact same thing live to far less brilliant audiences.

Of course, sometimes we think we have understood when we realling
haven't (see people who think they "got" OOP after playing a bit with
C++, for example) and that is when discusions become really complicated.

So I should really say I *think* I have understood Keith's efforts,
except for one detail: in a system that is about automation and
contributions from everyone there seems to be a lot of manually edited
scripts by single authors. The explanation is that I am probably
thinking about things backwards (in terms of which script calls which
script), but that is a another thread.

Andreas Raab wrote:
> I can't speak for the board, only for myself. From my perspective this
> is an issue of mislabeling. When I see something called "3.11" I expect
> this to be about Squeak 3.11, the software, not a set of tools. I'm also
> a little at a loss why then, if this is all about tools, so many noises
> are made about the 3.11 release artifact. If the entire goal of the
> process is to be able to load these tools into arbitrary versions, then
> why don't we just load them into the trunk and get over it? There is
> clearly still a level of disconnect somewhere but I'm hoping it's not
> quite as drastic as the last one.

This was caused by a series of steps in which things got renamed. The
whole thing started when the previous board announced "Squeak 3.x is
dead, 4.0 is Spoon" and Keith reacted saying he had already put in a lot
of effort for a process for future 3.x versions and that others felt the
same and would fork if needed. After discussions, the board approved
Keith's project as an official 3.11 with 4.0 now being just a clean
license version of 3.x and Spoon renamed 5.0 (the current board has not
discussed the latter at all).

So Keith was just saying "3.x forever!" (consistent with his proposal)
while the board used the 3.11 name.

Since we are having this discussion on #squeak-dev, it is important for
the community to know that each board member can have their own opinions
but then we reach a common decision. For example, I would far prefer to
see Squeak move to a binary based development model (I would mention
Projects and Etoys here) than the current source based things we are
doing (trunk, bob or whatever). But if the community likes Monticello,
that will get my vote.

-- Jecel


Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Re: 3.11 and the trunk

Colin Putney
In reply to this post by Andreas.Raab

On 19-Aug-09, at 10:15 AM, Jecel Assumpcao Jr wrote:

> For example, I would far prefer to
> see Squeak move to a binary based development model (I would mention
> Projects and Etoys here) than the current source based things we are
> doing (trunk, bob or whatever).

Forgive me for seizing on a throw-away comment like this, but would  
you mind expanding on this a bit? Are you saying you prefer something  
spoonish, where CompiledMethods  are passed directly from image to  
image? Something else?

Colin

Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Re: 3.11 and the trunk

Göran Krampe
In reply to this post by Andreas.Raab
Hi!

Andreas Raab wrote:

> Göran Krampe wrote:
>> - Keith has more or less said that the trunk effort is counter
>> productive in comparison to his/their 3.11 effort. Personally I might
>> think it is a gray area, but even so - perhaps once and for all - with
>> a pedagogic mindset - explaining to us all how this is so and also
>> describing "trunk" and "bob" in some way that all of us can grok it...
>> would let us come to some conclusion.
>
> It's quite simple really. Software development works by developers
> working together in a shared repository, right? That's the trunk. The
> trunk is nothing but a shared repository that the developers have commit
> rights to. In the course of software development, builds happen on a
> regular basis. That's Bob. Bob is a tool to assemble various bits of
> code and content and compile this into a result. Simple as that.
>
> Mantis role in this process with regards to code is to assemble
> contributions from non-core-developers in the project; people who are
> not given direct commit rights to the repository. These contributions
> need to be reviewed and integrated. That's Installer for you. The result
> of the integration process goes back into the repository. There's the
> trunk again. All the parts have their place in the process.

I really, really would like to hear Keith comment on the above because
if it is indeed like you describe it- then I am at a total loss over the
whole hoopla. I suspect it is not as simple as you describe above.

>> At this point I don't really give a rats about the above! :) I just
>> want us to heal, embrace, drink beer and find a way to move on. This
>> "war" has hurt us I think, I always brag about Squeak/Smalltalk being
>> a very friendly place, but it has slipped a bit lately. And thinking
>> that it might have been my posts setting off the fire doesn't really
>> make it feel better ;)
>
> I'm not unhappy about it. We need to get moving again. Whatever triggers
> this process is a good thing. And I never expected the process to be
> painfree, but I do think we are making progress if only by illustrating
> that other models of contributions can actually be successful. Now we
> have to work out how to make this work together to produce an actual
> piece of software called Squeak.

True.

>> BTW, can we please get our current status/process "trunk" or "bob" or
>> whatever we are using these days written down on squeak.org in an
>> official manner? Perhaps this is already done.
>
> I have nothing on Bob but I think the most relevant links are these:
>
> http://installer.pbworks.com/Squeak311Proposal
> http://board.squeak.org/2009/07/02/a-new-community-development-model/

Right, and neither is really on www.squeak.org. I know Keith couldn't
understand why his 3.11 stuff needed to be on www.squeak.org - but it
really makes it much more official and easy for anyone to find.

>> This also goes for the 3.11 team - does it even exist anymore?  Let us
>> get our act together.
>
> I leave it to Matthew to respond to this question.

Right.

regards, Göran

PS. ...now will focus on DS hacking instead...


Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Re: 3.11 and the trunk

Ian Trudel-2
2009/8/19 Göran Krampe <[hidden email]>:
>> http://installer.pbworks.com/Squeak311Proposal
>> http://board.squeak.org/2009/07/02/a-new-community-development-model/
>
> Right, and neither is really on www.squeak.org. I know Keith couldn't
> understand why his 3.11 stuff needed to be on www.squeak.org - but it really
> makes it much more official and easy for anyone to find.

I agree with you, Göran. Moreover, it should no longer be a "proposal"
if it takes any official position in the community. The proposal
should be revamped, adapted to the new direction taken, and clarified
before making it to the official web page.

Ian.
--
http://mecenia.blogspot.com/

Reply | Threaded
Open this post in threaded view
|

[squeak-dev] binary development (was: 3.11 and the trunk)

Jecel Assumpcao Jr
In reply to this post by Colin Putney
Colin Putney wrote on Wed, 19 Aug 2009 14:25:21 -0700:

> On 19-Aug-09, at 10:15 AM, Jecel Assumpcao Jr wrote:
>
> > For example, I would far prefer to
> > see Squeak move to a binary based development model (I would mention
> > Projects and Etoys here) than the current source based things we are
> > doing (trunk, bob or whatever).
>
> Forgive me for seizing on a throw-away comment like this, but would  
> you mind expanding on this a bit? Are you saying you prefer something  
> spoonish, where CompiledMethods  are passed directly from image to  
> image? Something else?

Heh, I got asked about this on IRC as well. Though I had actually
started to explain this a little in the original email, I ended up
deleting it to keep on topic. With a new subject line I don't feel I
have to worry about that. Some details about this (with a few drawings)
can be found in the Chunky Squeak wiki page:

http://wiki.squeak.org/squeak/584

The idea is to be more like the Etoys users which can load binary
projects containing not only the code they need but also hand crafted
objects which have no source (like a drawing, some nested Morphs or even
some text). This is very simplistic compared to Spoon, and my proposal
was even more simplistic. In particular, this doesn't handle the case
where any changes to bytecodes or object format are needed.

-- Jecel


Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] binary development (was: 3.11 and the trunk)

Igor Stasenko
In reply to this post by Colin Putney
2009/8/20 Jecel Assumpcao Jr <[hidden email]>:

> Colin Putney wrote on Wed, 19 Aug 2009 14:25:21 -0700:
>> On 19-Aug-09, at 10:15 AM, Jecel Assumpcao Jr wrote:
>>
>> > For example, I would far prefer to
>> > see Squeak move to a binary based development model (I would mention
>> > Projects and Etoys here) than the current source based things we are
>> > doing (trunk, bob or whatever).
>>
>> Forgive me for seizing on a throw-away comment like this, but would
>> you mind expanding on this a bit? Are you saying you prefer something
>> spoonish, where CompiledMethods  are passed directly from image to
>> image? Something else?
>
> Heh, I got asked about this on IRC as well. Though I had actually
> started to explain this a little in the original email, I ended up
> deleting it to keep on topic. With a new subject line I don't feel I
> have to worry about that. Some details about this (with a few drawings)
> can be found in the Chunky Squeak wiki page:
>
> http://wiki.squeak.org/squeak/584
>
> The idea is to be more like the Etoys users which can load binary
> projects containing not only the code they need but also hand crafted
> objects which have no source (like a drawing, some nested Morphs or even
> some text). This is very simplistic compared to Spoon, and my proposal
> was even more simplistic. In particular, this doesn't handle the case
> where any changes to bytecodes or object format are needed.
>

The central question, which arising immediately is, what is the
credible way(s) to reproduce such artifacts?
When we having a source code, we could (re)compile it on a different
system. But what you propose to do with pure binary data, a soup of
objects, in respect that it is incredibly hard to understand, what
bits you need and what's not, in case if you need to do clean-up ,
refactor, rewrite and simply analyze what is happening.
This is what making a huge difference, for instance, between
applications with open source code and applications shipped in binary
form - you can only report bugs, but can't realy make any suggestions
about what happening.
I don't think that developers of Squeak should be victims of such situation(s).

> -- Jecel
>



--
Best regards,
Igor Stasenko AKA sig.

Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] binary development (was: 3.11 and the trunk)

Eliot Miranda-2
Hi Igor,


On Wed, Aug 19, 2009 at 6:00 PM, Igor Stasenko <[hidden email]> wrote:
2009/8/20 Jecel Assumpcao Jr <[hidden email]>:
> Colin Putney wrote on Wed, 19 Aug 2009 14:25:21 -0700:
>> On 19-Aug-09, at 10:15 AM, Jecel Assumpcao Jr wrote:
>>
>> > For example, I would far prefer to
>> > see Squeak move to a binary based development model (I would mention
>> > Projects and Etoys here) than the current source based things we are
>> > doing (trunk, bob or whatever).
>>
>> Forgive me for seizing on a throw-away comment like this, but would
>> you mind expanding on this a bit? Are you saying you prefer something
>> spoonish, where CompiledMethods  are passed directly from image to
>> image? Something else?
>
> Heh, I got asked about this on IRC as well. Though I had actually
> started to explain this a little in the original email, I ended up
> deleting it to keep on topic. With a new subject line I don't feel I
> have to worry about that. Some details about this (with a few drawings)
> can be found in the Chunky Squeak wiki page:
>
> http://wiki.squeak.org/squeak/584
>
> The idea is to be more like the Etoys users which can load binary
> projects containing not only the code they need but also hand crafted
> objects which have no source (like a drawing, some nested Morphs or even
> some text). This is very simplistic compared to Spoon, and my proposal
> was even more simplistic. In particular, this doesn't handle the case
> where any changes to bytecodes or object format are needed.
>

The central question, which arising immediately is, what is the
credible way(s) to reproduce such artifacts?
When we having a source code, we could (re)compile it on a different
system. But what you propose to do with pure binary data, a soup of
objects, in respect that it is incredibly hard to understand, what
bits you need and what's not, in case if you need to do clean-up ,
refactor, rewrite and simply analyze what is happening.
This is what making a huge difference, for instance, between
applications with open source code and applications shipped in binary
form - you can only report bugs, but can't realy make any suggestions
about what happening.
I don't think that developers of Squeak should be victims of such situation(s).

    it is possible to have your cake and eat it too.  One can create a binary format that includes source and includes the meta-source for its creation.  But including a binary representation allows much faster loading, loading without a compiler, and source hiding if one choses not to include the source.

There are other advantages, such as not cluttering up the changes file when one loads a package  In the VW parcel system, to which I added source management, we replaced the SourceFiles with a SourceFileManager whose job was to manage the sources and changes file and an arbitrary number of source files for parcels, the binary format.  In the parcel file the source pointers of compiled methods are the positions of their source in the parcel source file.  When one loads a parcel the SourceFileManager adds the file to its set of managed files and assigns an index for the source file.  The parcle loader then swizzles all the source pointers so that they include the source file index along with the position.  So accessing the source for a method loaded form a parcel accesses that parcel's source file.  We used a floating-point like format for source pointers, where the exponent was the source file index, and the mantissa was the position in the file.

We didn't create a single file format, having two separate files for binary and source, which is probably a mistake.  A format with a short header, followed by source, followed by binary, followed by metasource, would be easier to manage than three separate files.

We didn't include any metasource, but we did include pre-read, load and unload actions.  I did a very bad job on version numbering and prerequisite selection.

That's not the whole story but enough to start answering your question.  If there is a well-defined definition of the objects in a package and that definition is included in the package as metasource, then one can comprehend the binary package's contents by examining the metasource and can reproduce creating the package, provided that the tools are careful to impose ordering, etc.

best
Eliot
 

> -- Jecel
>



--
Best regards,
Igor Stasenko AKA sig.




Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] binary development (was: 3.11 and the trunk)

Igor Stasenko
2009/8/20 Eliot Miranda <[hidden email]>:

> Hi Igor,
>
> On Wed, Aug 19, 2009 at 6:00 PM, Igor Stasenko <[hidden email]> wrote:
>>
>> 2009/8/20 Jecel Assumpcao Jr <[hidden email]>:
>> > Colin Putney wrote on Wed, 19 Aug 2009 14:25:21 -0700:
>> >> On 19-Aug-09, at 10:15 AM, Jecel Assumpcao Jr wrote:
>> >>
>> >> > For example, I would far prefer to
>> >> > see Squeak move to a binary based development model (I would mention
>> >> > Projects and Etoys here) than the current source based things we are
>> >> > doing (trunk, bob or whatever).
>> >>
>> >> Forgive me for seizing on a throw-away comment like this, but would
>> >> you mind expanding on this a bit? Are you saying you prefer something
>> >> spoonish, where CompiledMethods  are passed directly from image to
>> >> image? Something else?
>> >
>> > Heh, I got asked about this on IRC as well. Though I had actually
>> > started to explain this a little in the original email, I ended up
>> > deleting it to keep on topic. With a new subject line I don't feel I
>> > have to worry about that. Some details about this (with a few drawings)
>> > can be found in the Chunky Squeak wiki page:
>> >
>> > http://wiki.squeak.org/squeak/584
>> >
>> > The idea is to be more like the Etoys users which can load binary
>> > projects containing not only the code they need but also hand crafted
>> > objects which have no source (like a drawing, some nested Morphs or even
>> > some text). This is very simplistic compared to Spoon, and my proposal
>> > was even more simplistic. In particular, this doesn't handle the case
>> > where any changes to bytecodes or object format are needed.
>> >
>>
>> The central question, which arising immediately is, what is the
>> credible way(s) to reproduce such artifacts?
>> When we having a source code, we could (re)compile it on a different
>> system. But what you propose to do with pure binary data, a soup of
>> objects, in respect that it is incredibly hard to understand, what
>> bits you need and what's not, in case if you need to do clean-up ,
>> refactor, rewrite and simply analyze what is happening.
>> This is what making a huge difference, for instance, between
>> applications with open source code and applications shipped in binary
>> form - you can only report bugs, but can't realy make any suggestions
>> about what happening.
>> I don't think that developers of Squeak should be victims of such
>> situation(s).
>
>     it is possible to have your cake and eat it too.  One can create a
> binary format that includes source and includes the meta-source for its
> creation.  But including a binary representation allows much faster loading,
> loading without a
> compiler, and source hiding if one choses not to include the source.
> There are other advantages, such as not cluttering up the changes file when one loads a package  In the VW parcel system, to which I added source management, we replaced the SourceFiles with a SourceFileManager whose job was to manage the sources and changes file and an arbitrary number of source files for parcels, the binary format.  In
> the parcel file the source pointers of compiled methods are the positions of
> their source in the parcel source file.  When one loads a parcel the
> SourceFileManager adds the file to its set of managed files and assigns an
> index for the source file.  The parcle loader then swizzles all the source
> pointers so that they include the source file index along with the position.
>  So accessing the source for a method loaded form a parcel accesses that
> parcel's source file.  We used a floating-point like format for source
> pointers, where the exponent was the source file index, and the mantissa was
> the position in the file.
> We didn't create a single file format, having two separate files for binary
> and source, which is probably a mistake.  A format with a short header,
> followed by source, followed by binary, followed by metasource, would be
> easier to manage than three separate files.
> We didn't include any metasource, but we did include pre-read, load and
> unload actions.  I did a very bad job on version numbering and prerequisite
> selection.
> That's not the whole story but enough to start answering your question.  If
> there is a well-defined definition of the objects in a package and that
> definition is included in the package as metasource, then one can comprehend
> the binary package's contents by examining the metasource and can reproduce
> creating the package, provided that the tools are careful to impose
> ordering, etc.
> best
> Eliot

I think you inevitably made wrong decisions, because you went this way
by allowing an
arbitrary binary data , held by package.
In such situations it is much more easier to make a mistakes.
But sure, one who's making no mistakes is one who doing nothing :)

Obviously one of the side of such problem is uniform object memory,
where each object could
reference any other object and limited only by a imagination of people.
There is no layers or any other means which could establish a certain
barriers (which we calling a modules)
in smalltalk.
It means, that once you integrated the parcel into image, and started
using it, you may have a hard times trying to unload it.
It is possible to develop an image as an artifact, which contains both
binary & sources , but such approach
having a drawbacks, which we, by the way, trying to overcome nowadays.
Practice shows that such approach is credible only
for a small group of individuals, but becomes a bottleneck if you
adopt such scheme for a wider community.

So, i think , that before entering this domain (allowing binary data),
first we should solve more basic problems of smalltalk & its design -
modularity, name spaces, layering & etc etc.. Only the we could return
to original question and solve it.

--
Best regards,
Igor Stasenko AKA sig.

Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] binary development (was: 3.11 and the trunk)

Jecel Assumpcao Jr
In reply to this post by Igor Stasenko
Igor Stasenko wrote on Date: Thu, 20 Aug 2009 04:00:32 +0300:
> The central question, which arising immediately is, what is the
> credible way(s) to reproduce such artifacts?
> When we having a source code, we could (re)compile it on a different
> system. But what you propose to do with pure binary data, a soup of
> objects, in respect that it is incredibly hard to understand, what
> bits you need and what's not, in case if you need to do clean-up ,
> refactor, rewrite and simply analyze what is happening.

All true, but that is also the case of the image. Which is why we had
that whole discussion with some Debian guys about Squeak not being free
since you can't generate the current image from text sources like you
can for Self or GNU Smalltalk.

I like to classify computer systems as "blueprint" on one extreme, where
each execution starts wth an empty state and a description of what is to
be done, and as "living" on the other extreme, where once execution is
started it continues forever and accumulates more and more information
not in the original description (you can cheat and interrupt execution
by dumping the full state and later restoring it - like with Squeak
images). Each one has its problems and its advantages. Lots of bugs
which don't matter much in blueprint systems (like memory leaks) are
fatal for living systems. The latter need sophisticated error detection
and correction mechanisms ("lint" for images) which we don't currently
have. On the other hand, developing on living systems is so much more
productive that I am willing to deal with their problems.

Note that PCs were pure blueprint systems when they only had floppy
disks and applications were simply run instead of installed. With
internal fixed disks they have moved more and more towards living
systems (Windows registry and many other things). Many people long so
much for the old days that they cope with the changing behavior of
Windows over time by throwing out their computers every two years and
buying a new one. Certainly such people would not be very happy with the
direction I would like Squeak to go.

> This is what making a huge difference, for instance, between
> applications with open source code and applications shipped in binary
> form - you can only report bugs, but can't realy make any suggestions
> about what happening.

All of the tools that created the bits in the first place, as well as
the tools to change them are inside the same image as the bits. So I
don't agree with your analogy.

Note that just because you call some bits inside the computer "text"
doesn't mean you can manipulate them without special tools. It is just
that if the list of such tools is large and include things as vi, emacs,
notepad and so on then the tool is invisible to you. But it is there. If
I give you some text I wrote on my ZX81 (odd Sinclair character set) or
on the Burroughs 6700 (EBCDIC) then these tools won't help you very
much.

> I don't think that developers of Squeak should be victims of such situation(s).

The original thread was about how I vote for things that will make
Squeakers happy, not for what I would personally like. So nobody will be
a victim because of me. But I see no harm in discussing these things -
most people aren't even aware that we are making a choice.

-- Jecel


Reply | Threaded
Open this post in threaded view
|

[squeak-dev] Re: 3.11 and the trunk

Andreas.Raab
In reply to this post by Göran Krampe
Göran Krampe wrote:

> Andreas Raab wrote:
>> Göran Krampe wrote:
>>> - Keith has more or less said that the trunk effort is counter
>>> productive in comparison to his/their 3.11 effort. Personally I might
>>> think it is a gray area, but even so - perhaps once and for all -
>>> with a pedagogic mindset - explaining to us all how this is so and
>>> also describing "trunk" and "bob" in some way that all of us can grok
>>> it... would let us come to some conclusion.
>>
>> It's quite simple really. Software development works by developers
>> working together in a shared repository, right? That's the trunk. The
>> trunk is nothing but a shared repository that the developers have
>> commit rights to. In the course of software development, builds happen
>> on a regular basis. That's Bob. Bob is a tool to assemble various bits
>> of code and content and compile this into a result. Simple as that.
>>
>> Mantis role in this process with regards to code is to assemble
>> contributions from non-core-developers in the project; people who are
>> not given direct commit rights to the repository. These contributions
>> need to be reviewed and integrated. That's Installer for you. The
>> result of the integration process goes back into the repository.
>> There's the trunk again. All the parts have their place in the process.
>
> I really, really would like to hear Keith comment on the above because
> if it is indeed like you describe it- then I am at a total loss over the
> whole hoopla. I suspect it is not as simple as you describe above.

 From my perspective it is. I didn't make this up. Replace Bob with ant
and I've described the product development process we use at Qwaq.
Literally.

Cheers,
   - Andreas

123