How can we deal with Message rot?

How can we deal with Message rot?


Hi all,

This is a plea from a bug tracker for some guidance.

People in the community know that code that does not
stay in the image suffers from bit rot.
That is it gets more and more incompatible with what
is there.

In my travels (thru code) I have come across a more
pernicious decay. It can be best described a message
rot.

Basically a shared message loses its meaning. Or to be
more exact is given different meanings by different
Classes. The classes writen at different times or by
different authors all disagree on the meaning and thus
lose their polymorphism.

People will not be surprised to hear that the example
I have in mind is the meaning of forwardDirection,
rotationDegrees and heading. And also position,
referencePosition and centerOfRotation.

You can look at the code and track implementers and
senders to see how bad this problem has gotten. Any
bug in there I want to fix seems to involve clearing
up the whole tangle.

My problem and the guidance I seek it this.  A fix
will break backward compatability.  It has to because
once a confusion has been introduced the patches for
code use part of the confusion to patch around the
more objectionable bugs.  

How do I get community approval? I need consensus
amoung programmers it affects; Acceptance of the fixes
by the release team (of whatever future version it is
ready for.); And acceptance of the new understanding
as a standard amoung future enhancers and maintainers
of squeak.

How is this best achieved?

Yours in curiosity and service, --Jerome Peace

Footnote: more details of the heading etc. confusion
can be found mantis

http://bugs.impara.de/view.php?id=5674

0005674: Why doesn't heading = forwardDirection +
rotationDegrees for all morphs all the time?



__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around
http://mail.yahoo.com 

Jerome, thanks for the great introduction!

The 3.10 release is going to do something things that make the problem
worse, and some things that make it better.

The problem is how to make a system consistent when it is made out of
lots of components and there is no longer a single monolithic image.
One of the big advantages of an image is that it makes it easier to
make sure that the system is consistent.  As Jerome says, code that is
not in the image doesn't get updated when the image changes.  You
can't do an "all senders" to discover that it needs updating.  And you
can't do an "all implementors" to look at all the methods to infer
invariants.  As more and more people work on a system, it gets harder
and harder to make it consistent.  Is it possible to have the same
level of consistency in a system built from many componets developed
by many people that you find in the orignial Smalltalk image?

No, it isn't.  A system built by a large group cannot be as consistent
as a team built by a small group.  However, it can be consistent
enough.

The 3.10 release is going to componentize Squeak.  See my description
of Pavel Krivanek's KernelImage at
http://www.cincomsmalltalk.com/userblogs/ralph/blogView?showComments=true&entry=3342635112

The smallest image will be little more that what is necessary to run
the compiler.  It will have a text-based UI, i.e. Morphic is a
loadable package.

if we keep on ignoring code that is not part of the image and make the
image very small, then Squeak will die.  So, a major part of the 3.10
release will be an improved testing strategy.  We will have a test
server and a standard set of tests.  All changes will have to pass the
tests, which will involve loading lots of packages.  So, if you make a
change and ignore the standard packages, your change is unlikely to be
accepted.  There will be an easy way to load all the standard
packages, so if you are working in a minimal image then you can make
sure your changes will pass the tests by loading all the packages and
testing the changes in an expanded image.

Part of splitting the image into components involves having "official"
and "unofficial" components.  The official components are the ones
that the release team will update.  We'll have tests for them and make
sure that changes do not break them.  There will be a way for you to
get a component accepted as official, basically by providing a test
suite and having it pass all the existing tests.

This doesn't really answer your question of how to make a fundamental
change to Morphic.  Morphic is short on automated tests, so changes to
Morphic are more likely to  cause trouble even though they pass the
tests than changes to kernel classes.  But the idea is the same as for
any other system.  Write tests that show what you want to do, and then
make the changes, continually checking that you don't break anything
else.  Because Morphic is undertested, you'll probably have to write
some tests for Morphic to prove to yourself (and us) that you didn't
break anything.  These will be valuable beyond your particular
changes.

Most people seem to agree that Squeak needs to be modularized.  I
think that most of us do not realize the difficulties this will cause.
 The integrity of Squeak has been enhanced by its lack of modularity,
and we do not want to lose its integrity by modularizing it.  I think
that modularizing Squeak will lead to more tools to balance the
tradeoffs between componentization and consistency.

My goals as leader of the 3.10 release team are to modularize Squeak
and to develop processes for making a release that will improve the
quality of the software and make it easier to make a release.
Improved testing will be a key part of this.  I imagine that our first
few attempts will show a lot of need for improvement, and that we will
have to shift directions a few times until we come up with a good
process.  But these are the main things on my agenda.  The final 3.10
release will have lots of improvements, including new packages, new
tools, and new features.  But for me, these will be icing on the cake,
since my main goal is to figure out how to modularize Squeak without
losing the advantages of an integrated image.

-Ralph Johnson

On Dec 27, 2006, at 6:36 , Jerome Peace wrote:
>  more details of the heading etc. confusion
> can be found mantis
>
> http://bugs.impara.de/view.php?id=5674
>
> 0005674: Why doesn't heading = forwardDirection +
> rotationDegrees for all morphs all the time?

As far as I know, rotation and animation (that is, LOGO-style turtle  
semantics) is only seriously used in Etoys. For most other UI users  
it's merely a gimmick, so you would not need to worry about backwards  
compatibility too much. Etoys uses a very small set of messages  
(manifest in the Player class). As long as you ensure this small  
protocol does not break, Etoys should be fine.

Now, Player is very careful to always figure out the "right" morph to  
apply rotations to, depending on whether it wants to be "flexed" (in  
a TransformMorph) or if it handles rotation itself etc. This is why  
you see inconsistent behavior in the individual morph classes - in  
Etoys they are not used directly, but through the Player, and this is  
why your equation does not hold for each individual morph. I think  
you would have to eliminate TransformMorph completely, which might  
complicate other Morphs - until now, that "ugly" transformation stuff  
was factored out.

- Bert -

heading = forwardDirection + rotationDegrees (was Re:
How can we deal with Message rot?)

Hi Bert,

Thanks for your thought on this.
LOGO-style turtle  
>semantics) is only seriously used in Etoys.

I think it only seems this way.

First, I am making the assumption that by eToys you
are refering to programming with tiles, Players etc.

The larger realm of eToys also consists of morphs and
their behavior (particular behavior in response to
halo handles).  So what affects programming with tiles
spills over into morph considerations.

 
>For most other UI users  
>it's merely a gimmick, so you would not need to worry
about backwards  
>compatibility too much.

Ok. thanx.



>Etoys uses a very small set of messages  
>(manifest in the Player class). As long as you ensure
this small  
>protocol does not break, Etoys should be fine.
>
>Now, Player is very careful to always figure out the
"right" morph to  
>apply rotations to, depending on whether it wants to
be "flexed" (in  
>a TransformMorph) or if it handles rotation itself
etc.

Etoys player would benifit greatly from knowing less
about how a morph carries out its tilting
responsibilities. That means establishing a consistent
language and having all classes (and their programmers
and maintainers) stick to it.

This is the task of an oft promised but long overdue
refactoring of morphic stuff.

I haven't worked out the details but the notion I have
is that it should be possible to say

"player assuredFlexibleCostume heading: aHeading "  

and have the right thing happen.


And maybe later
"player assuredSimplestCostume"

which would remove redundant renderers.
 

>This is why  
>you see inconsistent behavior in the individual morph
classes.

But it shouldn't be that way.  The responsibilities
can be clearly defined and every morph would benifit.


Many of the deep bugs I run into have the
TransformationMorph decorator as their root.
Lots of the gribble leaving problems come from a
mismatch of assumptions between TfMorph ( which
believes you can translate the origin into the 3rd
quadrant) while every body else believes that truncate
is the same as floor  because all points are in the
1st quadrant. Clipping boxes get calculated wrong and
are often of by one just when you need them not to be.


Results: screen gribbles abound.  

>- inEtoys they are not used directly, but through the
Player, and this is  
>why your equation does not hold for each individual
morph. I think  
>you would have to eliminate TransformMorph
completely, which might  
>complicate other Morphs - until now, that "ugly"
transformation stuff  
>was factored out.

As I look at the code I see constant attempts to worry
about which morph is actually being talked to renderer
or renderee. The code practically begs to be
refactored. Its just a big job at the moment.
I await enlightenment and inspiration. Meanwhile I am
trying refactorings that lay the ground work for the
simplification of the Tf mess.

Yours in service, --Jerome Peace

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around
http://mail.yahoo.com 

On 2006 December 29 01:27, Jerome Peace wrote:
<<snip>>
> Many of the deep bugs I run into have the
> TransformationMorph decorator as their root.
> Lots of the gribble leaving problems come from a
> mismatch of assumptions between TfMorph ( which
> believes you can translate the origin into the 3rd
> quadrant) while every body else believes that truncate
> is the same as floor  because all points are in the
> 1st quadrant. Clipping boxes get calculated wrong and
> are often of by one just when you need them not to be.

Jerome,

Could you point out a few concrete examples (class, method) of this behaviour?
I started work on a (exteremely limited) set of tests for etoys and morphic,
and perhaps it would make sense to document what you found in a test;
although documenting something that is (if I understand) considered broken
may not be as valuable,  it should be easy to convert the test code if/once
the code under test is refactored.

Thanks Milan

Ralph,

I wish you much success.  Your idea of defining behavior by tests is a
good start.  Good tests for Morphic will be hard to write, and there
will obviously need to be a process for getting them (and any) tests
accepted for inclusion in the official suite, which will need to accept
additions, deletions, and edits over time.  Beyond that, some things
will need to break.  Sometimes, it will be that a test fails when it
should pass (overly aggressive assertions, etc.).  In other areas (e.g.
stream exhaustion), Squeak behaves very differently from other
Smalltalks, and probably should be changed.  I would further encourage
you to draw an underscore in the sand at the same time.

Modularization will involve some pain.  When it's over, I would hope to
see reasonable ANSI/cross-dialect compatibilty (including streams), and
unrestricted use of underscores in source code and the compiler.  Please
note that the latter does not have to kill single-key assignment, which
can be accomplished by optional editor behavior.

I doubt there will ever be a better time to clean up loose ends.

Bill



Ralph Johnson wrote (much snipped):
Part of splitting the image into components involves having "official"
and "unofficial" components. The official components are the ones
that the release team will update. We'll have tests for them and make
sure that changes do not break them. There will be a way for you to
get a component accepted as official, basically by providing a test
suite and having it pass all the existing tests.

This doesn't really answer your question of how to make a fundamental
change to Morphic. Morphic is short on automated tests, so changes to
Morphic are more likely to cause trouble even though they pass the
tests than changes to kernel classes. But the idea is the same as for
any other system. Write tests that show what you want to do, and then
make the changes, continually checking that you don't break anything
else. Because Morphic is undertested, you'll probably have to write
some tests for Morphic to prove to yourself (and us) that you didn't
break anything. These will be valuable beyond your particular
changes.

My goals as leader of the 3.10 release team are to modularize Squeak
and to develop processes for making a release that will improve the
quality of the software and make it easier to make a release.
Improved testing will be a key part of this. I imagine that our first
few attempts will show a lot of need for improvement, and that we will
have to shift directions a few times until we come up with a good
process. But these are the main things on my agenda. The final 3.10
release will have lots of improvements, including new packages, new
tools, and new features. But for me, these will be icing on the cake,
since my main goal is to figure out how to modularize Squeak without
losing the advantages of an integrated image.





Wilhelm K. Schwab, Ph.D.
University of Florida
Department of Anesthesiology
PO Box 100254
Gainesville, FL 32610-0254

Email: [hidden email]
Tel: (352) 846-1285
FAX: (352) 392-7029

Andrew P. Black skrev:
>
> I keep on thinking that Morphic is really elegant and that all of this
> will one day make sense if only I can learn to squint in the right
> way.  But I'm seriously beginning to wonder ...
Morphic has it's flaws... No, once you start scratching the surface of Morphic all sorts of stuff
starts to show up. Morphic transformations were retrofitted and this was kept trough the
ages as no one found it compelling enough to refactor Morphic to handle  
transformations without using the TransformationMorph stuff.

Tweak is next generation Morphic, and is written to handle
transformation from the ground up, but it has less users and less
documentation etc...

Karl