[squeak-dev] Perl is to CPAN as Squeak is to (what)?

>>>>> "Keith" == Keith Hodges <[hidden email]> writes:

Keith> The last thing we need is yet another package management system, when we have
Keith> hardly started using the features of the existing ones.

I think this says more about the existing ones (including Sake/Packages) than
about the desire for something that works.

Keith> Oh yes it is all very well to go and drum up political support for
Keith> various ideas and complain that it will not handle the needs of
Keith> millions of coders, but the reality is that it is a lot of hot air!
Keith> (see next paragraph)

If you build it, they will come.

Nobody was *clamoring* for the CPAN before it existed.  But once it did,
people started finding it easier to share their modules, and use other
people's modules.

I'm looking for that kind of ease.  Where I can say:

  "Here is a new parser generator I just created.  it requires an XML
  package.  I think it runs on 3.9 but I've only tested it on 3.10."

and then publish it somewhere.

And then you, coming along, can say:

  "I need a parser generator.  Oh, here's one Randal wrote.  I'll
  install it... oh it looks like it needs an XML parser, so that's
  getting installed too."

And then later, I can say:

 "Oh, I have an update, and it requires a newer version of the XML parser."

And then you can notice that update in some tool, and press a button, and you
get both the newer version of the XML parser and my parser generator.

When it's *that* simple, I think 90% of the people will be happy.

This isn't the goal of Sake/Packages.  So Keith, stop derailing this
conversation with what *you* want to do with Sake.  I'm not talking about
configuration management.

I'm talking about simple "publish" and "subscribe" options.

Make it as easy as listed above, including scaling so that 20 people can all
"publish" in the same 5-minute window without knowing a darn thing about each
other, and you'll have what I'm setting as the goal.  Your Packages isn't
meant for that.  Squeakmap is a lot closer, but is missing a few things,
perhaps borrowed from Universes.  We just all need to come together to make
this happen.

--
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
<[hidden email]> <URL:http://www.stonehenge.com/merlyn/>
Smalltalk/Perl/Unix consulting, Technical writing, Comedy, etc. etc.
See http://methodsandmessages.vox.com/ for Smalltalk and Seaside discussion

Randal L. Schwartz a écrit :
> Make it as easy as listed above, including scaling so that 20 people can all
> "publish" in the same 5-minute window without knowing a darn thing about each
> other, and you'll have what I'm setting as the goal.  Your Packages isn't
> meant for that.  Squeakmap is a lot closer, but is missing a few things,
> perhaps borrowed from Universes.  We just all need to come together to make
> this happen.
>

maybe what SqueakMap is missing is first-class actual objects for
package description. I mean that saying "this package Y loads into 3.9"
should be done by instanciating some kind of VersionCompatibility object
(the structure of which I have no real ideas about) and "this package Y
requires package X" would instanciate a PackageFetcher for X, which
along with the VersionCompatibility of both X and Y and the
PackageFetcher of X would figure out how to intall the whole thingy.

to say it differently, packages and their interactions should be made
first-class citizens of Squeak, so that the packages themselves can
reason about how and where they can be installed.

of course I have no clue about the actual implementation :) this is just
a sketch..


Stef

On Fri, Jun 27, 2008 at 2:12 PM, Keith Hodges <[hidden email]> wrote:
[snip]
>
> The last thing we need is yet another package management system, when we
> have hardly started using the features of the existing ones.

What we need is a package management system that works, *period*.  We
do not need a set of competing package systems that do not quite work.

> Oh yes it is all very well to go and drum up political support for various
> ideas and complain that it will not handle the needs of millions of coders,
> but the reality is that it is a lot of hot air! (see next paragraph)
>
> The [hidden email] list was recently re-rejuvenated to
> address this very issue, and as far as I am aware we had 1 more member
> subscribe to it as a result (and one member left). Hardly a sign of a
> community desparate to actually contribute to make things better for all, do
> you think?

I don't think so.  I think that it is a sign of a community that has
given up on the current packaging systems.

> Furthermore, the goals chosen by the current informal release team is to
> work in ways which will support the disparate communities that are emerging
> based upon squeak. (olpc etoys, sophie, croquet etc) Part of this involves
> championing tools that have been designed to actually work in all squeak
> derivatives.

Squeak errs trying to please all, but that is another discussion.
[snip]
> I think we could re-write SqueakMap to be a front end manager of
> Sake/Packages. If we did this then the dependencies could be managed to
> resolve to the correct data entries. But having said that, who has the time
> to actually do the work.

This, I think, is the crucial problem: who has the time to do the
work?  And having multiple competing systems with no clear best option
just makes matter worse.  Who, say, will work on SqueakMap if it is
going to be replaced by Universes? Who will work in Universes if it is
going to be replaces by Sake packages? And so on...

Hopefully, this thread will result in a project with a leader to see
it through, and not disipate like so many other things in squeak-dev.

For my part, I will try to contribute.  Although my Smalltalk skills
are still pretty limited (and my time much more), I could, for
example, contribute to reimplementing  SquekMap's interface in
Seaside.

Best Regards,

Victor Rodriguez.

Non profit organization is looking after a motivated fighting spirit
able to understand/resolve problems of/merge
- Monticello 1.0 1.5 2.0 ...
- Universe
- ChangeSet
- DeltaStream
- UpdateStream
- SqueakMap
- Sake
- Installer
- SAR
preferrably with good UI skills, both Morphic and Seaside,
experience of rock solid distributed services,
and an overall understanding of source code databases.

Knowledge in ImageSegment and/or Spoon-Naiad is a plus.

Unlimited gratitude rewards. Please send your expectations.

---------------

Given the skill level/salary ratio offered for the job, really
surprising if we cannot get a candidate...

Common cowards, anyone interested by the gauntlet?

---------------

Facing complexity, tentation of rewriting from scratch is high.
But didn't proliferation of good ideas already went out of control?
Please let us not invent a new one!

Have we other choice than progressing in SMALL STEPS?
Goran's ones seems reasonnnable / constructive...
Under the GSOC auspices maybe?
If only the pre-requisite list was not that frightening...

Any other dream should be reserved to solid financially supported teams.

Nicolas

El 6/27/08 5:38 PM, "nicolas cellier" <[hidden email]> escribió:

I plan to have this in SqueakLightII (aka. Unofficial 3.11)
> Have we other choice than progressing in SMALL STEPS?

I afraid no, so or we do a small step or we be always is same place

> solid financially supported teams.

Well, you could have more people  working in Argentina.
At same Euro fare you could have 4.5 and a same U$S fare 3.1.
I have so many talented students going to do COBOL (they need eat you know
?)

Edgar

I do not think there is need for another package management system, I just think that there should be an agreement on using and expanding _one_, incorporating what is best about all the others. I realize that means stepping on many, many toes and legs and probably even heads, but I think things need to be streamlined a bit.

If push comes to shove, I would even say, lets ditch them all and just use SVN like the rest of the planet (if that is possible). It is hard enough to sell a image-based language with a real IDE to the C-style crowd, the package  management systems should not add their grain of salt to the soup.
 
--
Claus Kick

"Wenn Sie mich suchen: Ich halte mich in der Nähe des Wahnsinns auf.
Genauer gesagt auf der schmalen Linie zwischen Wahnsinn und Panik.
Gleich um die Ecke von Todesangst, nicht weit weg von Irrwitz und Idiotie."

"If you are looking for me: I am somewhere near to lunacy.
More clearly, on the narrow path between lunacy and panic.
Right around the corner of  fear of death,
not far away from idiotism and insanity."

On Jun 27, 2008, at 2:50 PM, Randal L. Schwartz wrote:

this aspect of cpan ( and ruby's gem system and some other systems. )
is just so nice. i don't have to care that package A depends on 15 other
packages, it tells me that when i want to install and gives me the  
choice
install them or not. i hit yes to a few dependencies and bam! i have  
something
that works 99% of the time.

do the same thing by hand, ugh... so much time.
The simplicity of this basic operation is the killer feature. You miss
it so much when you go to systems that don't handle all of it for you.

If you haven't used something like it, you don't know what you are  
missing.
I speak as a convert who for years after CPAN was introduced continued
to build by hand because I was obstinate. I've seen the light,
all hail cpan, ruby gems, apt-get and all their brethren that 99%
of the time, work and save me tons of time.

On 28-Jun-08, at 5:27 AM, Claus Kick wrote:

> If push comes to shove, I would even say, lets ditch them all and  
> just use SVN like the rest of the planet (if that is possible). It  
> is hard enough to sell a image-based language with a real IDE to the  
> C-style crowd, the package  management systems should not add their  
> grain of salt to the soup.

Been there, done that... <shudder/>

Monticello was created because this turned out not to be feasible in  
practice. The simple fact is that Smalltalk *is* image based, and  
that's a feature, not a bug. For those who can't accept that, there's  
Ruby.

Colin

Colin Putney wrote:
Can you say something more about that? A couple of weeks ago I saw a
demo at HPI in Potsdam where students used SVN down to the method level,
and it seemed to me that this approach might very well work because the
SVN granularity is the same as the in-image granularity. It may also be
interesting that this wasn't even trying to deal with source files of
any sort - it retained the nature of the image and simply hooked it up
directly with SVN. From my perspective this looked like an
extraordinarily interesting approach that I am certain to try out as
soon as it is available.

Cheers,
   - Andreas

> On 28-Jun-08, at 5:27 AM, Claus Kick wrote:
>
> > If push comes to shove, I would even say, lets ditch them all and  
> > just use SVN like the rest of the planet (if that is possible). It  
> > is hard enough to sell a image-based language with a real IDE to the  
> > C-style crowd, the package  management systems should not add their  
> > grain of salt to the soup.
>
> Been there, done that... <shudder/>

Trying to get SVN to work or trying to sell Smalltalk to the C people, or both? :)

> Monticello was created because this turned out not to be feasible in  
> practice.

Could you please elucidate the main obstacles to this approach?

>The simple fact is that Smalltalk *is* image based, and  
> that's a feature, not a bug. For those who can't accept that, there's  
> Ruby.

I agree, in principle - though I would have liked slls to stay outside the image, that makes changing them externally much easier.
Instead that meant, unloading, reloading and sometimes crossing your thumbs and hoping all would go well and not fry your image.

--
Claus Kick

"Wenn Sie mich suchen: Ich halte mich in der Nähe des Wahnsinns auf.
Genauer gesagt auf der schmalen Linie zwischen Wahnsinn und Panik.
Gleich um die Ecke von Todesangst, nicht weit weg von Irrwitz und Idiotie."

"If you are looking for me: I am somewhere near to lunacy.
More clearly, on the narrow path between lunacy and panic.
Right around the corner of  fear of death,
not far away from idiotism and insanity."

Am 28.06.2008 um 21:45 schrieb Andreas Raab:


Do you think it would be feasible to exclusively manage an image from  
SVN sources?

The reason I'm asking is related to the "image" problem I reported  
earlier, the Linux folks demand the image to be to bootstrapped from  
sources + media files. Which IMHO would be a major re-engineering  
effort. E.g.,

http://lists.laptop.org/pipermail/devel/2008-June/015868.html

which is even one of the nicer emails in the thread, there were openly  
hostile posts, too.

- Bert -

On Sun, Jun 29, 2008 at 01:02:59AM +0200, Bert Freudenberg wrote:
> Do you think it would be feasible to exclusively manage an image from  
> SVN sources?
>
> The reason I'm asking is related to the "image" problem I reported  
> earlier, the Linux folks demand the image to be to bootstrapped from  
> sources + media files. Which IMHO would be a major re-engineering  
> effort. E.g.,

Well, it's not a great fit with Squeak, but Smalltalk/X has always worked
this way (and probably Gnu Smalltalk too), so clearly it's possible.

But frankly I suspect that a good old fashioned update stream with
human-readable change sets applied to some known base system would
address most of the perceived problem. If the base system consists
of sources plus "media" (a well-recognized image of known heritage),
then everything applied subsequently is easily traced, and can be
reapplied by anyone interested in doing so.

I would also note that those annoying Linux folks might just have a
point here. If we had followed these guidelines consistently over the
last few years, we would not have ended up with the mess of lost
author initials, untraceable changes, and unidentified licensing that
we are faced with today.

I think that Edgar has talked about trying to rebuild Squeak 3.9/10
from changes on top of a solid 3.8 image. I think he has the right
idea: fully traceable sources, all in plain text, and easily rebuilt
from a known base system. The 3.8 image itself was built from update
streams all the way back to an image of known heritage and license
status. Really, this is not a bad state of affairs.

Dave

El 6/28/08 11:43 PM, "David T. Lewis" <[hidden email]> escribió:

> I think that Edgar has talked about trying to rebuild Squeak 3.9/10
> from changes on top of a solid 3.8 image. I think he has the right
> idea: fully traceable sources, all in plain text, and easily rebuilt
> from a known base system. The 3.8 image itself was built from update
> streams all the way back to an image of known heritage and license
> status. Really, this is not a bad state of affairs.
>
> Dave

Thanks Dave.

Also I wish have a "Class repository" for ages.
See what I got from 3.8 in 2005

ftp://squeakros.atspace.com/squeakros.atspace.com/CompressedClasses/

password is squeak


I just build SqueakLightII aka Unofficial 3.11.
I plan to have the same thing here.

Its 3.9 with more packages going to live outside the image.
About 1400 clases, 2/3 size normal 3.10
All the way until 3.10 is with the same we do in Ralph era, but in easy to
follow .cs form.

The image could be updated hitting the load updates of Squeak flap or doing
Utilites slUpdates from Workspace.

At the moment you should reach 7183 with one error when image try to load
the Matthew version of Monticello 1.6 beta.
Close the error and repeat the loading and then save as 7183

I add a page on swiki, all could put any you wish

http://wiki.squeak.org/squeak/6056

And a couple of ready to run

In a Workspace,

|sourceFile |
sourceFile := HTTPLoader default retrieveContentsFor:
'http://wiki.squeak.org/squeak/uploads/6056/LightWeb.sar'.
sourceFile := RWBinaryOrTextStream with: (sourceFile content).
SARInstaller new fileInFrom: sourceFile

|sourceFile |
sourceFile := HTTPLoader default retrieveContentsFor:
'http://wiki.squeak.org/squeak/uploads/6056/ SqueakRosDev3dot10.sar'.
sourceFile := RWBinaryOrTextStream with: (sourceFile content).
SARInstaller new fileInFrom: sourceFile

Not to bad having a server and web page builder and a development set for a
pre-alpha ...

Help and we could have a polished system ready for ESUG or Smalltalks 2008


Also at the moment I development crude tools for analyze all in 3.10
Universes, the first output I send to "Discussion about development of
Squeak 3.10" <[hidden email]>

As seems Colin finished Monticello2, I wish try it if someone say the files
I should load from http://source.wiresong.ca/mc/

Edgar

> I would also note that those annoying Linux folks might just have a
> point here. If we had followed these guidelines consistently over the
> last few years, we would not have ended up with the mess of lost
> author initials, untraceable changes, and unidentified licensing that
> we are faced with today.

  At least this time around, the licencing part isn't question.  We
are making progress^^;

  BTW, there was a discussion about a month ago (I basically read them
just recently), and Bert was asking that how hard it is to do
bootstrap from source.  I know many of you have thought about the
actual bootstrapping.  That would go something like:

  - Write a compiler in another language.  That can generates the bits
    that are same as CompiledMethods.  For a class definition, it
    creates (yes) the network of pointers.

  - The compiler sticks the class definitions, method dictionaries,
    subclass structure, and compiled methods into a big "int*" array.
    The goal is to make that something just run, so for example, the
    stuff managed by ClassDescription (instanceVariable names and
    organization) don't have to be compiled.  Stuff like the source
    pointer is not needed at this stage.

  - There should be a support for IdentityDictionary, Array, Symbol
    and String. and for the latter two, a way to create instances
    would be needed.  SystemDictionary and Undeclared would be the
    instances of IdentityDictionary that should be created and gets
    into the int* array.  The symbol table that manages the instances
    of Symbol is needed during the compilation, but it probably
    doesn't have to go into the int* array, as we can recreate it in
    the next initialization stage.

  - During the compilation, we may have some garbage, but of course we
    just leave them in the int* array.  A class may get many
    subclasses.  The subclasses array of the class may be assigned
    many times, but we just cr

  - In this senario, nil, true, false aren't needed, I believe (as
    CompiledMethods don't hold onto them). However, it would be nice
    to "create" them early and store in the int* array.

  - A few more necessary instances that go to special objects array is
    created.  Perhaps only Processor and active context would be
    needed.

  - The int* array is saved into a file with the header.

  - Now launch the image with a regular VM.  The active context goes
    over a list of classes and initialize them.

  - Compile all sources again so that they get proper source pointer.
    (Or, scan the sources and set the source pointers properly).

  - Create a project, enter and save the image normally.

  - Load more code. and keep it going.

  How feasible is it to do?  Or, do somebody have some idea that how
many contexts would we need to create "manually"?  What other
instances are needed?  How much code rewriting would be necessary?

  Yes, there are other ways of doing it.  For example, if we make the
kernel part is fairly small (like 500k), we can have a program that
holds onto the bytes as constant and write out to disk.  This would be
enough to persuade these people.  We can load all the rest onto it
later.  (But if this is the case, I'd ask myself that if 300k is ok
why not 20MB?  That was the artificial distinction Andreas mentioned,
I believe.)

-- Yoshiki

Am 29.06.2008 um 04:43 schrieb David T. Lewis:

No it would not. The main issue for them is that you have to start  
from what they perceive as "binary blob" which is monkey-patched into  
newer versions.

- Bert -

Pavel Krivanek
http://www.comtalk.eu/Squeak/98

On Sun, Jun 29, 2008 at 4:09 AM, Yoshiki Ohshima <[hidden email]> wrote:

The compiler doesn't have to be in another language.  What is
important is that the compiler builds a Squeak image from a text file.
 The compiler can be written in Smalltalk as long as it uses only the
class definitions in the test file, not in the compiler's image.

In face, it would be possible to reuse the existing compiler to build
up a standard structure of classes in the image, but to put them all
in their own namespace.  Thus, an Object class defined in the test
file would not be the same as the Object class used in the compiler.
Thus, a system that bootstraps images from text files could first read
the text file and produce a set of classes in the image, and then
write out an image that contains only those classes.

Pavel Krivanek's KernelImage does the second part, but not the first.
It creates a parallel class hierarchy by copying part of the existing
classes, then it writes out an image that contains only those classes.

So, I think you could make a bootstrapping compiler by hacking the
existing compiler to read in a regular .st file into a separate
namespace.  Then you'd use KernelImage to write out the image.  The
system would have to make sure that instances of String,
CompilerMethod, etc., pointed to the new classes and not the old, but
KernelImage has to do something like this already.

Given KernelImage, I don't think that the compiler is all that hard to
write.  The hard part would be making the text file of all the
classes.  If I were doing this, I'd hack KernelImage to create the
text file for me, so I could at least create something similar to
KernelImage.

-Ralph Johnson

i havent read all of this yet, but the first thing that comes to my
mind is, why do we have so many different tools?  It seems like there
should be one way, Meaning squeakMap, or something else, to download
files.  I think keeping all of the methods to get local code in is
important though.

On 6/28/08, Claus Kick <[hidden email]> wrote:

--
David Zmick
/dz0004455\
http://dz0004455.googlepages.com
http://dz0004455.blogspot.com

>>>>> "Bert" == Bert Freudenberg <[hidden email]> writes:

Bert> No it would not. The main issue for them is that you have to start  from what
Bert> they perceive as "binary blob" which is monkey-patched into  newer versions.

The C compiler would fit the same definition, by that reasoning.

At some point, you have toggle switches.  Everything after that
is binary blobs building other binary blobs.  Do they really need
us to trace it back to toggle switches?

--
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
<[hidden email]> <URL:http://www.stonehenge.com/merlyn/>
Smalltalk/Perl/Unix consulting, Technical writing, Comedy, etc. etc.
See http://methodsandmessages.vox.com/ for Smalltalk and Seaside discussion

On 29-Jun-08, at 5:31 AM, Ralph Johnson wrote:

Alejandro Reimondo's Fenix stuff read (through some extraordinarily  
'creative' and Windows dependant code) source snippets from files and  
built a parallel object hierarchy and then wrote an image out via a  
modified tracer. It's a very simple idea but rather less simple to  
implement correctly. It is however certainly doable.

In principle one could certainly keep a tree of SVN like form, read in  
the code whilst traversing the tree, create objects to suit and then  
trace only those objects to create an 'image created from source'.

And you know what? If we put together a system to do that, there would  
still be objectors; there will always be objectors.

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Base 8 is just like base 10, if you are missing two fingers.

tim Rowledge wrote:

> And you know what? If we put together a system to do that, there would
> still be objectors; there will always be objectors.

If you can't edit it in emacs, it's not OpenSource.
At least that seems to be the acid test for some people...

So unless we support the four step magic dance ritual
- edit in emacs
- make config
- make
- make install
we're out by default.
After that probably because it's not Python...

Michael