[Packages] Split-Join in development universe etc

I added Split-Join as a package in the development universe.

This is a small set of functionality, much discussed on the list, which
has not yet gained the notoriety required to make it into 3.10. Making
this an item that can be named as a dependant, should other packages
make use of it this has the benefit of recording its usefulness. It also
means that I/you can write packages so that we can be sure it is loaded
when needed.

I proposed enabling Universes to run arbitrary, or Installer based
scripts, but this is considered somewhat cavalier. So undaunted, and no
less barbaric I have used a PackageInfo-#preamble to load this script!
This has the disadvantage of being hard to maintain and it is working
invisible magic behind the scenes , whereas the script option would have
been visible and manageable.

To use this you will need a PackageInfo-Base which supports preamble
scripts etc. I am not sure if the Monticello in 3.10 can support this
fully (it should do but I haven't tested it).

So in addition I have also uploaded Monticello1.5 ;-) to the package
universe, in some areas it has been a radical makeover of 1.0 This
should now be ready to use in 3.10 now that I have reinstated traits
support.

recent work includes
- fixed monticello configurations
- fixed preambles/postscripts
- fixed browsing of configurations
- wired up load/merge/browse buttons for configurations
- real version numbering 1.2.3b etc
- out of order loading
- doesnt loose methods from packages so easily as 1.0
- support for traits reinstated
- (untested diffs support not expected to work)

The monticello in 3.10 is unique among monticello versions in that I
find it impossible to work around so as to get it to load monticello
1.5, so there is a Monticello15-bootstrap package which loads it in from
source the first time. (Another task which a script would have been
helpful for)

A warning though, there is one significant show stopper bug to catch,
which I havent had time to track down in the past 3 months. When given a
package in which classes change their instVars significantly (i.e in
ways that 1.0 could not handle at all) the class can get corrupted and
that package needs recompiling. We have found this is needed
occassionally (i.e.it is usually signalled by the image crashing)

In spite of this I think that Monticello1.5 is nearly ready to be used
in 3.10. I tried using the old one for a few hours and got very
frustrated with loosing class extensions.

enjoy

Keith

On 8/1/07, Keith Hodges <[hidden email]> wrote:

Hi Keith,

it sounds really cool. Does it load on 3.9 too or only on 3.10? If
yes, my next dev images will include your Monticello1.5. If not, I
will have more work.

Thank you for your work

Bye

2007/8/1, Keith Hodges <[hidden email]>:

--
Damien Cassou

Damien Cassou wrote:
> Hi Keith,
>
> it sounds really cool. Does it load on 3.9 too or only on 3.10? If
> yes, my next dev images will include your Monticello1.5. If not, I
> will have more work.
>
> Thank you for your work
>
>  
Hi damien, I am using MC1.5 :-) in 3.8 and 3.10, so... 3.9 should work.

Keith

Keith Hodges <[hidden email]> writes:
> I added Split-Join as a package in the development universe.

What does it do?  You do not say.


> I proposed enabling Universes to run arbitrary, or Installer based
> scripts, but this is considered somewhat cavalier. So undaunted, and
> no less barbaric I have used a PackageInfo-#preamble to load this
> script! This has the disadvantage of being hard to maintain and it is
> working invisible magic behind the scenes , whereas the script option
> would have been visible and manageable.

The development package universe should be a place where we work
together.  The above does not suggest a working-together sort of
attitude.


Can you say what is wrong with Package Universes dependencies, that
you need to apparently add a new kind of dependencies?  What is it
that you want to happen in preamble scripts?


-Lex

Keith, did you miss my questions? :)

Jason Johnson wrote:
>  
>
>
>
>     Cool.  This is the double dispatch way that lets you do things
>     like:    #(252 211 203) joinWith: #(255 254) => #(252 255 254 211
>     255 254 203)
>     right?
>
yes thats the one. Because those methods are in a change set with discussion etc. managed
mantis. Loaded as per

Installer mantis ensureFix: 4874.

>
>         In spite of this I think that Monticello1.5 is nearly ready to
>         be used
>         in 3.10. I tried using the old one for a few hours and got very
>         frustrated with loosing class extensions.
>
>
>     Losing class extensions?  You mean star methods?
>
yes, there are two types of class extensions supported by monticello.
'*' prefixed extensions and '*' prefixed and '-override' suffixed method
categories.

Having monticello preserve the integrity of packages which either use
these, or have these used on them is difficult, but I think it is possible.

When monticello is not reliably supporting these extensions I would have
to have an image specifically for managing each package. I.e. to ensure
that the package I was managing was not over-ridden, and all packages
that it extends are currently loaded.

Now the monticello 1.5 ;-) orphanage keeps hold of methods and classes
which extend packages not currently loaded, and loading an overridden
method, loads the original, then the override, so that searching for the
original on saving should work.

cheers

Keith

Ok, thanks for you answers (comments below)

Of course it would be best, but this is a kernel extension, which is
under the control of the release team, not I.

I discovered that, while waiting for such things to to be integrated
into the kernel, I have found myself writing my code to be as generic as
possible instead.

As an example: I find myself writing code which avoids using split/join
until split/join is a generically available feature. This may be a long
wait for it to be added to 3.10, but it will never be added to 3.9, 3.8
or 3.7 so.

If I keep on doing this, it means that all efforts to improve the kernel
are wasted, because even if I write a kernel improvement, I am
conditioned by the working context to not use such extensions until they
are deemed acceptable  by the consensus, which in the case of "Message
eating Null", or Gorans name space improvements is probably never.

This has been bugging me for a while so, I figure the only way I can see
to get around this is to legitimize the idea of a kernel-patch in the
universe (which formalizes which versions the patch is applicable to)
and have it managed properly in mantis, i.e. feedback discussion. new
versions etc.

With this in place I feel that I can then get on and use it without
shame. If it is found that lots of essential packages make use of that
patch then this would be a good case for getting it included by the
release team.

just my 2p

Keith

On 8/7/07, Keith Hodges <[hidden email]> wrote:

On Aug 7, 2007, at 10:00 PM, Jason Johnson wrote:

> I was under the impression that now is the best time ever to get  
> needed things into the image.  The "Message eating null" and  
> namespaces are big controversial things that can't be ignored once  
> present.  But this package seems like a "no-brainer" since it's  
> just method adds and they put in something that seems glaringly  
> missing from Squeak at the moment.  If you have tests and things  
> you should still try I would say, there might still be time.

I wouldn't call it a no-brainer. In fact, if it was up to me, I  
wouldn't add these methods to the base image, for two reasons:

1. They're not idiomatic Smalltalk. If you need to join a collection  
of strings into a single string with a separator it's easy to do with  
#streamConents: and #do:separatedBy:. If you find you're doing it so  
often that you want something more succinct, that's a code smell.

2. The base image should be getting smaller, not bigger. Split-join  
can live quite well as an external package.

I've been asked about the lack of #join: a few times by people coming  
to Smalltalk from Ruby or Python. They were incredulous that such a  
fundamental thing was missing. But in Smalltalk it's not fundamental  
- Smalltalk is object-oriented, not string-oriented.

Colin

Colin Putney wrote:
+1 on all accounts.

Cheers,
   - Andreas

>  were incredulous that such a fundamental thing was missing. But in
> Smalltalk it's not fundamental - Smalltalk is object-oriented, not
> string-oriented.
>
> Colin
>
. This implementation of split/join does not operate on strings, it
operates on collections of objects

. It is no less idiomatic than some of the more esoteric methods in
collection

. The whole point of putting something like this into the kernel is to
raise the level of abstraction a bit. In doing so it is totally possible
that some other code somewhere else can use this more succinct form and
the net image size is reduced.

For example: notice the tidy use of readable smalltalk and typical
idioms in NetNameResolver-readDecimalByteFrom: and #addressFromString:

regards

Keith

> I was under the impression that now is the best time ever to get
> needed things into the image.  The "Message eating null" and
> namespaces are big controversial things that can't be ignored once
> present.
But in actual fact in these two cases they CAN be ignored once present.

However I happen to think that it would be useful if they were present,
Null in particular.

For example I use Object-#log as a handle to a ProcessLocalLog instance
which plugs in to the logging technology of your choice (toothpick
/simple log etc). In moving towards the kernel image ideal, all of the
Transcript stuff in the core could be ripped out and use "self log
kernel: message" instead, the base implementation of which can return
Null if no logging package is loaded.

As an excercise I think that it is interesting that if you cant get two
or three methods of the importance and usefullness of split and join
into the image (and btw unless I am mistaken there is no existing idiom
for split) then you cant get anything in there, unless of course you
happen to be the one in charge.

cheers

Keith

On 8/8/07, Colin Putney <[hidden email]> wrote:

On 8/8/07, Keith Hodges <[hidden email]> wrote:

No need to take it personal, the jury is still out on these things (even the join/split that I thought most people would like).  But saying "oh just put it in, you can totally ignore it once it's in!" is definitely not the way to get votes imo.

 
>
> And after suggesting we can just ignore these classes, here comes a
> suggestion to start using it in Object! :)
fair point. :-)

> That's not a bad idea about logging, but in this case I think about
> what other places do.  In Pier you can have persistence, or you can
> turn it off by setting NullPersistence.  That is, of course, a
> "message eating null" class but it is explicit.  If they replaced that
> with "Null", then things get a little les clear, no?
In this particular case, it is not appropriate to use Null instead of
NullPersistence, because NullPersistence still does something, well a
lot of things actually. It is not a true Null, its a 'manage your data
but in the image only'  provides the default 'manage your data'
implementation for many functions. When using the
Pier-PersistencyManager to switch between persistency schemes, switching
from PRMagmaPersistency to PRNullPersistency requires actions to pull
the data in from the db into memory in order for NullPersistency to be
selected. In some cases the PRNullPersistency simply reflects back
double dispatch ed requests, whereas its subclasses do something (see
#realizeKernel). PRNullPersistency way of "doing nothing" is to Null's
way of "doing nothing", as a Mirror is in comparison to a Black Hole,
they just dont work the same way.

If we do it the other way around, and implement a "clear", "specific"
NullLogger class, then we have added a class to the image that has only
one use. The next requirement for a Null class has no choice but to
implement its own bespoke solution. If we use Null as the NullLogger
then, we have a choice.
> I'm not yet convinced that this comes up enough to be a kernel class.
I think it could do if you are ripping things out, since it might become
an option for replacing those ripped out things.
I would also like to argue for the inclusion of #askFor: as another aid
for interacting with items that are optionally present.
> Of course one can always code in ways to *make* something come up a
> lot, but with current usage I can think of about 3 places in the whole
> image it comes up, and each is made more clear imo by having an
> explicit class.
>
>     for split) then you cant get anything in there, unless of course you
>     happen to be the one in charge.
>
Just a little dig at the process that has continued to demonstrate that
democracy doesn't seem to work in terms of pushing innovations, but
appears to favour the status quo
> No need to take it personal.
As a suggestion for 3.11+ my proposal is to provide the tool for users
to specify, packages, fixes/changesets, removals. With such a tool, we
ask community members to lay out their own road maps of items they would
like to see in the release. For example: kernel-events-logging-solution,
kernel-streams-replaced-with-nile, nile-streams-backward-compatibility,
openGL-rendering, exupery -support, morphic3, system-editor,
atomic-monticello etc etc.

This encourages forks of 3.10 as each person can design their own idea
of 3.11p (p for personal) and in their working environment develop the
pieces that they wish to contribute.

Ralph and Edgar really didn't like this multiple-forks idea, however the
point I think that was missed is, that if all the forks and pieces of
fork are defined in the same build tool, then they can be designed with
integration in mind. The aim of forking is not to go off to develop your
own squeak, but to be able to manage your branch of innovation in such a
way as to be able to contribute it back into the main stream in the mid
to long term, and to have it potentially visible, and potentially
programmed into the release schedule (via the unstable branch) so that
others in their personal-unstable-build-based-branches can integrate
their pieces with yours.

I tried using a wiki for this tool in order to promote collaborative
working, but people did not seem to take to the idea. So instead I
propose using a monticello package(s) to specify the parts/dependencies
and the build processes. Doing it this way gives us the versioning, and
merging tools of monticello to collaboratively design the release build
processes.

Then we collate a 3.11a, from the contributed roadmap ideas, into a
stable and an unstable build process, and individually we can work on
the pieces, until each piece is in a state that is ready to move from
the unstable build into the stable build. Pieces that are not yet ready
at the end of a given release-time-box, can simply be passed into the
next release unstable build process. When the relase-time-box is
completed for a release 3.11a-stable gets nominated as 3.11b, and only
bug fixes are accepted. Meanwhile 3.12a begins adopting the unused
pieces from the 3.11a-unstable branch into a revised roadmap definition.

Thus bug-fix integration becomes just one piece of the process, rather
than consuming the whole effort as it seems to now. This piece can be
delegated to part of the team, while there are other proactive projects
are in progress.

just a few of my thoughts, and at the moment I have no time to work on
this for at least a month.

best regards

Keith

Jason Johnson wrote:
> Well, the interesting thing about this package is that it joins any
> collection with any other collection, it's not just strings.

It's really just strings. You're getting confused by the (sorry to say
that) poorly written code. It uses <SequenceableCollection> joinUsing:
<delimiter> which looks very general at first glance but once you dig
into it you'll notice that it really ought to be <delimiter> join:
<Collection> and that really only String is a meaningful delimiter here.
The implementation as it is creates a lot of clutter by doing it the
other way around and then (needlessly) double-dispatching into Character
and String. Besides which the code is very poorly formatted and not
documented at all(there isn't even a single comment anywhere despite
some highly unusual patterns). Not the kind of stuff I like to see in
the kernel.

> Wanting code to be concise is code smell?  If you meant adding a method
> to collections that only joins strings is a code smell then ok, but
> otherwise.... well I strongly disagree to say the least.

Concise can mean different things. If it means a bloated collection
interface to accommodate every single person's pet extensions then I
think that stinks. And that's exactly what we've seen over the last
years - ever-increasing amounts of partially overlapping functionality
none of which is widely used and only makes it more difficult to find
the few truly useful methods. Split for example, is for all practical
purposes completely subsumed by findTokens: so why do we need split in
addition to findTokens?

Cheers,
   - Andreas

Hi!

[SNIP]
I agree with Andreas here and my only comment is that renaming methods
(and using deprecation procedures of course) to a better name is on the
other hand a good thing to do - my guess is that people often don't find
findTokens: because neither of the two words "find" and "tokens" are
likely to be used when searching in method finder for example.

When it comes to "holy cows" like Collection for example there are a
multitude of aspects to consider:

- Backwards compatibility, worth breaking?
- Deprecation procedure, easy to cope with if we indeed change something?
New names likely to be found etc?
- ANSI and/or similarity with other Smalltalks, worth "doing as the
others" or worth doing it differently?
- Minimalism vs usefulness, keeping the standard protocols small but
powerful and not adding methods that are very seldom used (they can be
kept in extension packages).
- Quality. Changes in these classes need to be rock solid. And yes, if
they don't even have method comments then sorry, I would reject them on
that basis ALONE, because then the author surely hasn't tested or thought
it through enough if he can't bother to even write some small comments.

And while on the subject I still think #removeAll (my own pet Collection
addition that got shot down by some people a few years back) would be
useful :)

And finally - we probably do need a Collection hierarchy
overhaul/cleanup/improvement - but that should be a careful project. I am
not even convinced yet that Traits is a "really useful good thing" (given
discussions I have followed - no personal experience yet though) - I was
as enthusiastic as everyone when Nathanael first did his presentation at
OOPSLA a few years beack - but I (as I presume Andreas too) also would
like to see some solid concincing use of them.

They sound nice in theory, but so does a lot of things. In my experience I
am more and more favouring simpler solutions to problems that involve
fewer components and relations - even if they do indeed break a few
"golden rules" of OO or programming in general (like for example avoiding
redundancy like the plague or not allowing a sinle line of UI code into
domain objects etc).

Oops, this post turned longer than planned - sorry.

regards, Göran

El 8/9/07 5:32 AM, "Göran Krampe" <[hidden email]> escribió:

> I still think #removeAll (my own pet Collection
> addition that got shot down by some people a few years back)

Göran :

Nice see you again on list.

If you have Collection additions not in current 3.10, I wish know.
I could consult with Ralph and maybe have into Squeak again.

I bet several masters have nice things on his computers unknown to many of
us, is time to show, discuss and share.


Cheers.

Edgar