Dolphin Map?

Hello world(tm),

is anybody working/interessted in something like SqueakMap for Dolphin
Smalltalk?

If something like it would be available - which features would you like
to see in it?

I allready had some rough thoughts and came up with the following:

  * The general idea is to provide a repositry of package *information*
- The packages maybe still on other servers.
  * The server information is cached locally.
  * Each package has properties like name, comment, author, version
required DST version, prerequsites, further information ....
  * This information is searchable using a full text search.
  * You can define Package groups (e.g. packages which are normally used
together).
  * Each package can be downloaded directly or if the package is part of
an "meta-package" (e.g. a zip/exe installer) it's wrapper.
  * Each package information downloaded may contain some code to
actually install the package locally (unzipping, calling an exe)...
  * Download/Install is a seperate action. E.g. it's possible to sync
with DolphinMap (download all package information and the package
itself) without actually installing it.
  * Local Installer in two parts. First part needs to be installed from
a  PAC file and is more or less static (final). The second part gets
downloaded (with a local copy if this fails) to make sure to have the
most up-to-date package.
  * Local/Remote installer only needs OA provided packages - no 3rd
party package required.
  * All communication is done over http.


Nice to have:
  * Package upload/update from local image (I'm still trying to figure
out where to put the package then ... either DolphinMap is hosting
packages as well or it's just the information that get's published.
  * STS integration
  * Codelets: Parts of ST code which micht be helpfull but are not part
of a package.

At the moment I'm not quite sure whether it makes sense to use a
DB+Backend on the server or just a plain XML file which get's downloaded.

XML would be simpler but a DB with it's own ST Server provides greater
flexibility!?!?



Comments are highly appreciated.

CU,

Udo

On Wed, 08 Dec 2004 16:15:45 +0100,
Udo Schneider <[hidden email]> wrote:
> Hello world(tm),
>
> is anybody working/interessted in something like SqueakMap for Dolphin
> Smalltalk?

would be cool to have :-)

>   * Local Installer in two parts. First part needs to be installed from
> a  PAC file and is more or less static (final). The second part gets
> downloaded (with a local copy if this fails) to make sure to have the
> most up-to-date package.

I'd prefer to let the package handle the installation procedure as
far as possible. So a local watchdog for sniffing out conflicts before
they happen would suffice for one part. If the package needs to do
something else besides filing in (like the initial versioning with STS),
the package author is the one who knows how to do this, hence should
program it.

>   * STS integration

+1

Good night,
s.

Stefan Schmiedl wrote:
Maybe I got you wrong - this divided package approach was only intended
to be used with the DolphinMap package(s) itself. All other packages are
simply normal PAC packages.

Did I get you wrong?

CU,

Udo

On Thu, 09 Dec 2004 18:54:19 +0100,
Udo Schneider <[hidden email]> wrote:
ahh.. my bad. you got me perfectly right, but *I* got *you* wrong.
Sorry for the confusion.

s.

Udo Schneider wrote:

> Hello world(tm),

Hello Udo ;-)


> I allready had some rough thoughts and came up with the following:

I'd been thinking a little about the same thing, and (encouragingly!) had come
up with a fairly similar idea.  Some notes below (I've re-ordered your text a
bit).


>   * The general idea is to provide a repositry of package *information*
> - The packages maybe still on other servers.

I think that's the crucial idea.  Not a collection of code, but of links to,
and descriptions of, code.


>   * Local/Remote installer only needs OA provided packages - no 3rd
> party package required.
>   * All communication is done over http.

I agree, it wouldn't be so helpful if you had to install just about all the 3rd
party stuff that's available just to use the 'map'....


> At the moment I'm not quite sure whether it makes sense to use a
> DB+Backend on the server or just a plain XML file which get's downloaded.
>
> XML would be simpler but a DB with it's own ST Server provides greater
> flexibility!?!?

The problem of having (or needing) a centralised server is "who runs it?" (in
the sense of administers, as well as in the sense of providing machine space).
The way I'd been thinking decentralises the idea:

-) Anyone can put together an XML file that describes one or more packages.
They don't have to "own" the packages to do so.  Call these XML files "map
files" (not a very good name, but I can't think of anything better offhand
"dolphin-net" makes me think of tuna fishing...).  There's no special reason to
use XML, a more readable text format might be preferable.

-) Anyone can publish a map file on the Web, but they don't have to do so to
use it.  They could have a local map file just for their own use.

-) Map files can include "links" (with comments) to other such files anywhere
on the Web (or locally, I suppose).

-) Goodie authors would (we hope) like the idea, and would publish a map file
at a well-known location on their website.

Thus the on-the-Net representation of the map would /only/ be a collection of
interlinked map files.

That (if you like the idea) means that the local "client" is the only code that
exists (unless someone wants to build a mapfile editor -- actually, I had
rather imagined the local client as an "aggregator" of mapfiles, and as such it
would be natural for it to act as an editor for them too).

It would be up to the user of the local client to decide which map files s/he
wanted to include from off the Web.  So if, say, I had created a map file for
some old site that is still useful but not actively maintained, then the user
could choose whether to "believe" me or not, and could also ignore (that part
of) my map file if the referenced site vanished without me updating my links to
it.


>   * The server information is cached locally.

Do you mean "cached" exactly (i.e. a mere performance tweak), or that it's
/held/ locally and can be refreshed explicitly ?  Not a huge difference, but if
it's just to be a cache, then the caching built into URLMon library may be
sufficient by itself.


>   * Each package has properties like name, comment, author, version
> required DST version, prerequsites, further information ....

Copyright, licence,.... (and see below)


>   * You can define Package groups (e.g. packages which are normally used
> together).

Good idea.  Can groups overlap (I think they should be allowed to) ?

Maybe copyright and licence should be at this level.  (Both would be better, I
suppose).  Also at this level (or both) should be location of on-line
documentation (if any).


>   * Each package can be downloaded directly or if the package is part of
> an "meta-package" (e.g. a zip/exe installer) it's wrapper.

I like the idea of being able to download the stuff from within the client, but
I don't think that:


>   * Each package information downloaded may contain some code to
> actually install the package locally (unzipping, calling an exe)...

is a good idea.  Accepting and running code from off-the-web without looking at
it first is an absolute error, IMO, and I /certainly/ wouldn't use any tool
that forced me to do that (not if I could possibly avoid it).  I don't mind
having one or two pre-defined "how-to-install-me" actions, but (for instance)
running a .EXE is /not/ an acceptable action.  In fact the only predefined
action that I can think of is automatic unzipping into a defined location --
but even that would worry me unless the unpacking code was vetted /very/
carefully.


>   * Download/Install is a seperate action. E.g. it's possible to sync
> with DolphinMap (download all package information and the package
> itself) without actually installing it.

The way I'd thought of it, the local client wouldn't have any ability to load
packages at all; it would only fetch stuff off the web.  It's not that I think
that its a bad idea to be able to load packages straight from the map, but
doing so is considerably harder than just downloading (and means that the
client cannot be packaged as a separate .EXE tool).

I think we need a solution to several problems related to package management,
but it seems to me that trying to solve them at the same time as setting up The
Map would be over-ambitious -- at least for a first-cut.

Does that make any sense ?

If it /does/ make sense, and appeals to people, it sounds as if there's some
overlap with RSS, but I know nothing about that.  Is there something that it
would be easy to re-use ?  (My guess is no, re-invention is often quicker,
safer, and easier, than re-use -- but that may just be my prejudice)

    -- chris

Udo Schneider wrote:

> is anybody working/interessted in something like SqueakMap for Dolphin
> Smalltalk?

Saw this thread awhile ago and was also thinking of something...

One could easily work with an OPML ( xml outline file ) pointing to
various DOAP ( another xml format, description of a project ) file with
information about versions and package locations.

This would let joe-random-package maintainer to maintain the DOAP and
package files on their own webspace, so its distributed, all you'd need
is a base XML parser and some code to download the packages and load
them into the image...

Mark Derricutt wrote:

> One could easily work with an OPML ( xml outline file ) pointing to
> various DOAP ( another xml format, description of a project ) file with
> information about versions and package locations.

For the interested, the OPML spec is here (very readable):
    http://www.opml.org/spec

And DOAP is decribed here:
    http://usefulinc.com/doap

and there's a 4-part IBM dev-works article (which I haven't read yet, but they
are usually pretty good) starting here:
    http://www-106.ibm.com/developerworks/xml/library/x-osproj.html

=====

Personal opinions:

I can't see a lot of benefit in OPML, I don't think it gives us much, and --
although agreeably simple (an undeniable virtue)  -- it also look pretty stupid
to me.  E.g. /pixel/ offsets in a cross-platform spec ?! (OK, it's optional,
but...)  And why use RFC822 format dates (with 2-digit years) ?

DOAP looks, at first sight, like a pretty good match. Very close to what /I/
had in mind anyway, I don't know how much Udo (or any lurkers in this thread)
will like it.

I'll post again when I've read more about DOAP -- which looks as if it'll take
some time, everything in the XML world is so bloody /complicated/...

    -- chris

Chris Uppal wrote:

>I can't see a lot of benefit in OPML, I don't think it gives us much, and --
>although agreeably simple (an undeniable virtue)  -- it also look pretty stupid
>to me.  E.g. /pixel/ offsets in a cross-platform spec ?! (OK, it's optional,
>but...)  And why use RFC822 format dates (with 2-digit years) ?
>  
>
Most people I know seem to think that OPML is a realllllly stupid spec,
but its also a common format, although I'm sure a better one could be made.

>I'll post again when I've read more about DOAP -- which looks as if it'll take
>some time, everything in the XML world is so bloody /complicated/...
>  
>
I'm already using DOAP in my small Dolphin app for controlling automated
updates of my application. ( checks a URL and downloads the doap, if a
newer release exists - download and execute the newer installer ).

It shouldn't be hard to change this to load in some code....  mmmmm.


>    -- chris
>
>
>  
>

Chris Uppal wrote:
> I think that's the crucial idea.  Not a collection of code, but of links to,
> and descriptions of, code.
My intention was to not force anybdy from placing *his* code on a
central server.

> The problem of having (or needing) a centralised server is "who runs it?" (in
No problem. I have the server and I'll run it.

> -) Anyone can put together an XML file that describes one or more packages.
> They don't have to "own" the packages to do so.  Call these XML files "map
> files" (not a very good name, but I can't think of anything better offhand
> "dolphin-net" makes me think of tuna fishing...).  There's no special reason to
> use XML, a more readable text format might be preferable.
This distributed approach is really interessting. I decided against it
for the first relase to keep is simple/stupid (KISS).

The current file "format" is heavily "inspired" by the Smalltalk chunk
format.

> -) Anyone can publish a map file on the Web, but they don't have to do so to
> use it.  They could have a local map file just for their own use.
That's something I wanted to catch with multiple "sources" for these Ma
files. One of the sources would be the DM server over http.


> -) Map files can include "links" (with comments) to other such files anywhere
> on the Web (or locally, I suppose).
Yep. I even thougt about extending the definition of "Dolphin Map
Package" to include something like the News Archive / EC as well.

> Thus the on-the-Net representation of the map would /only/ be a collection of
> interlinked map files.
Agreed. see above.

> That (if you like the idea) means that the local "client" is the only code that
> exists (unless someone wants to build a mapfile editor -- actually, I had
> rather imagined the local client as an "aggregator" of mapfiles, and as such it
> would be natural for it to act as an editor for them too).
The Map file Editor is currently in development. I don't want to code
the files by hand ;-)

> It would be up to the user of the local client to decide which map files s/he
> wanted to include from off the Web.  So if, say, I had created a map file for
> some old site that is still useful but not actively maintained, then the user
> could choose whether to "believe" me or not, and could also ignore (that part
> of) my map file if the referenced site vanished without me updating my links to
> it.
I tried to catch this with the idea of multiple sources for map files.


> Do you mean "cached" exactly (i.e. a mere performance tweak), or that it's
> /held/ locally and can be refreshed explicitly ?  Not a huge difference, but if
> it's just to be a cache, then the caching built into URLMon library may be
> sufficient by itself.
It's downloaded locally and the client informs you if a newer version of
the map file is available at a specific source. This gives you the
possibility to download the most recent version.

> Copyright, licence,.... (and see below)
Good point

>>  * You can define Package groups (e.g. packages which are normally used
>>together).
> Good idea.  Can groups overlap (I think they should be allowed to) ?
Yes. They might overlap. I was thinking of something like debian
packages. They have requirements and "recommendations".

> Maybe copyright and licence should be at this level.  (Both would be better, I
> suppose).  Also at this level (or both) should be location of on-line
> documentation (if any).
I thought of appliying this information on both layers. Some
package-packages I know include packages not written by the same author
   - therefore license etc. on both levels - this means ability to save
space on one hand but flexibilty to have licenses/authors per package on
the other.

>>  * Each package can be downloaded directly or if the package is part of
>>an "meta-package" (e.g. a zip/exe installer) it's wrapper.
> I like the idea of being able to download the stuff from within the client, but
> I don't think that:
>
>>  * Each package information downloaded may contain some code to
>>actually install the package locally (unzipping, calling an exe)...
It's allways an option. Surfing through the packages available for
dolphin I found three "installation" methods:
  - Single package (.PAC) download
  - Exe self extracting file
  - ZIP Files

PAC files can be downloaded and installed directly.
EXE files may be started automaticaly (user choice)
ZIP files may be extracted by by your goodie/WinZip/PowerArchiver/Manually.

The user will allways have the choice to to everything *manually*.

> is a good idea.  Accepting and running code from off-the-web without looking at
> it first is an absolute error, IMO, and I /certainly/ wouldn't use any tool
> that forced me to do that (not if I could possibly avoid it).  I don't mind
No force. See above.

> having one or two pre-defined "how-to-install-me" actions, but (for instance)
> running a .EXE is /not/ an acceptable action.  In fact the only predefined
> action that I can think of is automatic unzipping into a defined location --
> but even that would worry me unless the unpacking code was vetted /very/
> carefully.
See above. The idea is to have a piece of code associated with every
package-package which gets several pieces of information in variables.
This block can do whatever you want - with the exeception that this
block as only access to certain classes (I still need to verify if this
is possible). E.h. the block can call the "UnZipper" - however this
"class" still let the user decide how the zip file should be extracted.

>>  * Download/Install is a seperate action. E.g. it's possible to sync
>>with DolphinMap (download all package information and the package
>>itself) without actually installing it.
> The way I'd thought of it, the local client wouldn't have any ability to load
> packages at all; it would only fetch stuff off the web.  It's not that I think
> that its a bad idea to be able to load packages straight from the map, but
> doing so is considerably harder than just downloading (and means that the
> client cannot be packaged as a separate .EXE tool).
My approach for the first version is different. Sorry ;-)

> I think we need a solution to several problems related to package management,
> but it seems to me that trying to solve them at the same time as setting up The
> Map would be over-ambitious -- at least for a first-cut.
Agreed - I don't want to save the world ... just make it a little bit
friendlier ;-)

> Does that make any sense ?
Yes :-)

> If it /does/ make sense, and appeals to people, it sounds as if there's some
> overlap with RSS, but I know nothing about that.  Is there something that it
> would be easy to re-use ?  (My guess is no, re-invention is often quicker,
> safer, and easier, than re-use -- but that may just be my prejudice)
RSS would be nice to publish new items in DM ... I'll think about it.

CU,

Udo

Mark Derricutt wrote:
> One could easily work with an OPML ( xml outline file ) pointing to
> various DOAP ( another xml format, description of a project ) file with
> information about versions and package locations.
I'll take a look at these specs - but be warned. The first release will
use "home-grown" config files.

> This would let joe-random-package maintainer to maintain the DOAP and
> package files on their own webspace, so its distributed, all you'd need
> is a base XML parser and some code to download the packages and load
> them into the image...
This distributed approach is really appealing ... maybe one could extend
  the idea of DM publishing a public map and links to other map files as
well.

CU,

Udo

Udo Schneider wrote:

> It's downloaded locally and the client informs you if a newer version of
> the map file is available at a specific source. This gives you the
> possibility to download the most recent version.

Out of curiosity, how are you checking for newer versions, in my own
update code I was originally doing:

  '3.0.10.2' > '3.0.10.1'

for a simple compare, but that failed the moment you did '3.0.10.12' >
'3.0.10.4'

So I came up with:

version: versionA isGreaterThan: versionB
    | subA subB partCount |
    subA := versionA subStrings: '.'.
    subB := versionB subStrings: '.'.
    partCount := subA size.
    1 to: partCount
        do:
            [:part |
            | numberA numberB |
            [subB at: part] on: BoundsError do: [:x | ^true].
            numberA := (subA at: part) asNumber.
            numberB := (subB at: part) asNumber.
            numberA > numberB ifTrue: [^true]].
    ^false

Which seems to work fine for all my tests.

>  '3.0.10.2' > '3.0.10.1'
>
> for a simple compare, but that failed the moment you did '3.0.10.12' >
> '3.0.10.4'
>
> So I came up with:

Mark, I got bored:


"Filed out from Dolphin Smalltalk XP"!

Object subclass: #Version
 instanceVariableNames: 'major minor revision build'
 classVariableNames: ''
 poolDictionaries: ''
 classInstanceVariableNames: ''!
Version guid: (GUID fromString: '{1797166A-A710-413A-96A3-262EC8064441}')!
Version comment: ''!
!Version categoriesForClass!Kernel-Objects! !
!Version methodsFor!

< anObject
 (major ~= anObject major) ifTrue: [ ^(major < anObject major) ].
 (minor ~= anObject minor) ifTrue: [ ^(minor < anObject minor) ].
 (build ~= anObject build) ifTrue: [ ^(build < anObject build) ].
 (revision ~= anObject revision) ifTrue: [ ^(revision < anObject
revision) ].
 ^false!

= anObject
 ^(anObject isKindOf: Version) ifTrue: [self equals: anObject] ifFalse:
[false]!

> anObject
 ^(self < anObject) not!

build
 ^build!

delimiter
 ^$.!

equals: anObject
 ^major = anObject major & (minor = anObject minor) & (build = anObject
build)
  & (revision = anObject revision)!

hash
 ^(major hash) + (minor hash) + (build hash) + (revision hash)!

major
 ^major!

minor
 ^minor!

nextNumberFrom: stream
 "returns either a number, or if its at end of stream, 0"
 ^(stream upTo: self delimiter) asNumber!

parseInput: aString
 | stream |
 stream := aString readStream.
 major := self nextNumberFrom: stream.
 minor := self nextNumberFrom: stream.
 build := self nextNumberFrom: stream.
 revision := self nextNumberFrom: stream.
!

revision
 ^revision! !
!Version categoriesFor: #<!public! !
!Version categoriesFor: #=!public! !
!Version categoriesFor: #>!public! !
!Version categoriesFor: #build!public! !
!Version categoriesFor: #delimiter!private! !
!Version categoriesFor: #equals:!private! !
!Version categoriesFor: #hash!public! !
!Version categoriesFor: #major!public! !
!Version categoriesFor: #minor!public! !
!Version categoriesFor: #nextNumberFrom:!private! !
!Version categoriesFor: #parseInput:!private! !
!Version categoriesFor: #revision!public! !

!Version class methodsFor!

from: aString

 ^(self new) parseInput: aString! !
!Version class categoriesFor: #from:!public! !




I also added a method to String, #asVersion:

String>>asVersion
    ^Version from: self.


now you can do:

'1.0.0.12' asVersion > '1.0.0.2' asVersion.