Squeak scripts in UNIX

Friends,

I would like to update the manual page of Squeak (its SCRIPTS section is
out of date). At least for the Debian packages.

Now, the Squeak script to be executed must be specified as URI, not an
ordinary path. That means that it is not possible to rely on the
traditional (and handy) feature of UNIX kernels that if the user tries
to execute a file that begins with `#' (shebang) character, the kernel
actually runs the program specified in that first line and appends the
filename that is actually executed as a last parameter to that program.

This is what everybody knows. However, if Squeak requires URI instead of
path, it is actually not possible to create Squeak scripts. Is this
true? What should be inside the SCRIPTS section in the manual page?

I know that using Squeak for scripts seems to be retrogressive, but I
have used it once and then it helped me much.

Work in progress (with mistakes) is here
http://altair.dcs.elf.stuba.sk/~kosik/tmp/squeakvm.html

The SCRIPTS section is empty, now. The original text is very nice and
simple, but also wrong.
----------------------------------------
Here is the most simple way how a Hello, world script could look like

f := FileStream fileNamed: '/dev/stdout'.
f nextPutAll: 'Hello, world'.
f nextPut: Character lf.
SmalltalkImage current snapshot: false andQuit: true

It, unfortunatelly, cannot be run normally via shebang but it must be
run as follows

squeak -vm display=null -vm sound=null $SQUEAK_IMAGE file://`pwd`/hello.st

Seems quite terrible. Isn't there a simpler way?
--
Matej Kosik

The goal is great.  Squeak users are only helped by having more
interfaces to the outside world.  Calling it backwards is just silly;
most of us would like a programming system we can use today, not in
some imagined future where Unixy systems are obsolete.  Unixy systems
are very popular right now, and so we should support interfacing to
them, including via the script-files form of executable.

A nit with your explanation, by the way: it's #!, not # by itself.
The # is a "hash", and the ! is a "bang", and thus #! is "hashbang"
which, now that you mention it, does sound like "shebang" :) .


On the technical side, feel free to propose a changeset that updates
the image however you need.  Is there any legacy we need to worry
about whatsoever?


As a general approach, a kind of header that works well would look
something like this:

  #!/bin/sh
  exec squeak -headless -- "$0" "$@"
  !#

What happens here is that the Unix kernel starts starts /bin/sh on the
script file, but /bin/sh sees only a single "exec squeak" command.
The arguments are "--", which tells the Debian squeak script not to
try and interpret any later arguments.

The advantages of this approach over using #!/usr/bin/squeakvm are:

  1. You can supply arguments to squeak if you
  want, e.g. -headless.  I am told this is non-portable
  on the #! line.

  2. You can find squeak via $PATH instead of needing it
  to be hardcoded.


This requires two changes in the standard Squeak image to work.

First, the above approach requires that the image allows files as
arguments, not just URL's.  I see no reason not to support both: if it
starts with "[a-z]+:", then it's a URL, othrewise it's a file.

Second, we need a way to support a block comment at the head of the
script.  In the example above I used #!...!#, which I like.  It
has a tiny bit of synergy in that it is what scsh and scala also
use for their script files.  However, we are free to do whatever
we want; it's not a big deal.

To implement #!...!# (or whatever else), the best solution forward
appears to be to have the script-running code in the image strip off
the header.  Technically it would be nicer if this was a valid
notation for comments in Squeak, but that's a language change and
would require negotiating with all of Squeakdom.



Anyway, again, go go go.  :) I would love to see better scripting on
Unix, and the first step to doing it well is a clean interface for
running these things at all.


Lex



PS -- There was a time in history when image files were directly
executable on Unix.  The trick is simply to put a proper #! header on
*image* files, and to make sure that the header is exactly 512 bytes.
I don't know why that change got backed out.  It was a simple
implementation and was convenient for Unix-based Squeakers.

Il giorno mar, 13/02/2007 alle 12.04 -0500, Lex Spoon ha scritto:
> This requires two changes in the standard Squeak image to work.
>
> First, the above approach requires that the image allows files as
> arguments, not just URL's.  I see no reason not to support both: if it
> starts with "[a-z]+:", then it's a URL, othrewise it's a file.
[SNIP]

> To implement #!...!# (or whatever else), the best solution forward
> appears to be to have the script-running code in the image strip off
> the header.

I worked on something like that at the last Camp Smalltalk in Prague.
Mine was just a hack of a couple of lines of code, but it worked pretty
well. The [a-z]+: change should be as easy as that.

> PS -- There was a time in history when image files were directly
> executable on Unix.  The trick is simply to put a proper #! header on
> *image* files, and to make sure that the header is exactly 512 bytes.
> I don't know why that change got backed out.  It was a simple
> implementation and was convenient for Unix-based Squeakers.

That was cool. I'd love to have that in again.

        Giovanni

Matej Kosik wrote:
> This is what everybody knows. However, if Squeak requires URI instead of
> path, it is actually not possible to create Squeak scripts. Is this
> true?

The relevant code is in ProjectLauncher>>startUpAfterLogin and is trying
to recognize regular files, or rather it is even assuming regular files
and figures out specific types of URLs:

   scriptName := (Smalltalk getSystemAttribute: 2) ifNil:[''].
   scriptName isEmpty ifFalse:[
     "figure out if script name is a URL by itself"
     isUrl := (scriptName asLowercase beginsWith:'http://') or:[
       (scriptName asLowercase beginsWith:'file://') or:[
       (scriptName asLowercase beginsWith:'ftp://')]].
     isUrl ifFalse:[scriptName := 'file:',scriptName]].

Looks like the culprit is the last line, trying to construct an
(incorrect) file url. For starters, try fixing this line and see where
it gets you.

Cheers,
   - Andreas

Lex Spoon wrote:
> First, the above approach requires that the image allows files as
> arguments, not just URL's.  I see no reason not to support both: if it
> starts with "[a-z]+:", then it's a URL, othrewise it's a file.

Yeah, right like in c:\my squeak\test.pr :-)

Cheers,
   - Andreas

Lex Spoon writes:
 > PS -- There was a time in history when image files were directly
 > executable on Unix.  The trick is simply to put a proper #! header on
 > *image* files, and to make sure that the header is exactly 512 bytes.
 > I don't know why that change got backed out.  It was a simple
 > implementation and was convenient for Unix-based Squeakers.
 
The 512 byte header space is still in the VM. I haven't tried using it
but I have seen the header code that skips past the first 512 bytes
recently.

Bryce

On Feb 13, 2007, at 19:33 , Andreas Raab wrote:

It actually works on Unix with an absolute path ;-)

- Bert -

On Feb 13, 2007, at 18:04 , Lex Spoon wrote:

> Is there any legacy we need to worry about whatsoever?

The only thing I can think of is when running as browser plugin, you  
should keep the "old" project launcher in place for that. The plugin  
will give an empty argument as script, so that's easy to distinguish.

The harder part is that there is no concept of a current working  
directory in Squeak. To find the script, you need to convert a  
relative path to the absolute one using cwd. This can either be done  
in the shell script, or one could abuse the SecurityPlugin which  
listens to environment variables ... or extend the VM ... or use  
OSProcess ...

- Bert -

Lex Spoon wrote:
[snip]
>
> A nit with your explanation, by the way: it's #!, not # by itself.
> The # is a "hash", and the ! is a "bang", and thus #! is "hashbang"
> which, now that you mention it, does sound like "shebang" :) .

I was not sure (I did not have books as hand) I just vaguely recalled
that there is a word for this concept.
You are right. #! is shebang. The first time I saw it was in the Minix
3rd Edition book (page 458, in bold). Also wiktionary contains a proper
password
http://en.wiktionary.org/wiki/shebang

This is partially why I like (also) English. It has words for many
subtleties.

>
>
[snip]
--
Matej Kosik

I do have a couple of refactoring AutoStart initiatives on the go.

The bad news being that the best code is sitting on a busted laptop at
the moment. Its ok its only the screen I think the HD is ok.

The current AutoStart code is a bit of a spaghetti monster, I tidied it
up quite a lot. However considering the legacy I was planning to hold
off on putting this forward until we have a testing infrastructure in place.

In the meantime I added some autostart functionality to Installer so
that installer and other classes can handle lots of weird and wonderful
parameters from a commandline start.

I am currently refactoring this out of Installer and into a separate
'Launcher' project and class.

Keith


               
___________________________________________________________
Now you can scan emails quickly with a reading pane. Get the new Yahoo! Mail. http://uk.docs.yahoo.com/nowyoucan.html

Lex Spoon wrote:
>   #!/bin/sh
>   exec squeak -headless -- "$0" "$@"
>   !#

An alternative which may not require changes to any parser is something
like the following:

  "exec" "squeak" "-headless" "--" "$0" "$@"

This is a pun of sorts: in Squeak, it's a sequence of five comments at
the start of the script; in shell, it's a sequence of words that happen
to be harmlessly redundantly quoted.

I've not tested that particular variant, but I have used a similar
punning trick in Scheme, where ';' introduces a single line comment:

  "true"; exec /path/to/scheme/interpreter "$0" "$@"
  (rest of scheme program)

To the shell, this is an execution of /bin/true followed by a
replacement of the current process by a scheme interpreter; to Scheme,
the first line is a self-evaluating string expression which happens to
have a comment on it, and the rest of the file is the actual program.

Regards,
  Tony

Andreas Raab <[hidden email]> writes:
> Lex Spoon wrote:
> > First, the above approach requires that the image allows files as
> > arguments, not just URL's.  I see no reason not to support both: if it
> > starts with "[a-z]+:", then it's a URL, othrewise it's a file.
>
> Yeah, right like in c:\my squeak\test.pr :-)

Heheh.  Well, then, how about [a-be-z]+:   :)


-Lex

Bert Freudenberg <[hidden email]> writes:
> The harder part is that there is no concept of a current working
> directory in Squeak. To find the script, you need to convert a
> relative path to the absolute one using cwd. This can either be done
> in the shell script, or one could abuse the SecurityPlugin which
> listens to environment variables ... or extend the VM ... or use
> OSProcess ...

On Unix, "FileDirectory default" gives you the Unix cwd.  And at any
rate, this is the "default" directory, so it is at least a defensible
place to start if you are given a relative path.


That said, that's very interesting that using the absolute path works
already.  Maybe the easiest thing, then, is to come up with some
shell-script magic to rewrite $0 into an absolute path.  Do any Unix
hackers here know how to do that?

-Lex

On Wed, 2007-02-14 at 15:12 -0500, Lex Spoon wrote:
> That said, that's very interesting that using the absolute path works
> already.  Maybe the easiest thing, then, is to come up with some
> shell-script magic to rewrite $0 into an absolute path.  Do any Unix
> hackers here know how to do that?
>
> -Lex

First of all let me say that I've been paying only the barest of
attention to this thread so I could be offbase here, and I'm not exactly
a shell guru...

#!/bin/sh
exec squeak -headless -- `pwd`"/$0" "$@"
!#

Not pretty, but I believe that would do the trick.  However I'm not sure
that pwd will work in the desired manner in all situations.  Also, some
logic is needed here to ascertain that $0 is not already an absolute
path.

Ken

On Feb 14, 2007, at 21:12 , Lex Spoon wrote:

> Bert Freudenberg <[hidden email]> writes:
>> The harder part is that there is no concept of a current working
>> directory in Squeak. To find the script, you need to convert a
>> relative path to the absolute one using cwd. This can either be done
>> in the shell script, or one could abuse the SecurityPlugin which
>> listens to environment variables ... or extend the VM ... or use
>> OSProcess ...
>
> On Unix, "FileDirectory default" gives you the Unix cwd.

No. It gives you the image directory.

> That said, that's very interesting that using the absolute path works
> already.  Maybe the easiest thing, then, is to come up with some
> shell-script magic to rewrite $0 into an absolute path.  Do any Unix
> hackers here know how to do that?

I was using this:

        case "$2" in
            /*) DOCUMENT="$2"
                ;;
            *) DOCUMENT="$PWD/$2"
                ;;
        esac

(from http://etoys.laptop.org/src/etoys.in)

- Bert -

On Wed, Feb 14, 2007 at 09:51:17PM +0100, Bert Freudenberg wrote:
There is a "current working directory" introduced into CommandShell,
but as Bert says is a concept that does not exist in Squeak, so it
had to be added. Furthermore, it works on a per-volume basis in Windows
(C: drive versus D: drive, etc), so it's not clear that this is a
very portable concept.

Dave

This has been tested on linux, but it looks like some help is needed to
get it working on mac osx.

Starting squeak like this:

#squeak Squeak3.9-final-7067.image http://installer.pbwiki.com/f/IBS.st 
Installer url=http://squeak310.pbwiki.com/Essential39Fixes 
SmalltalkImage save=squeak.image +quit

Note: When this script is running you will have to push the proceed
button manually for now.  This will give you an image that can do a
number of new things.

1. Fixes the scripts cant be local files bug.

#squeak -- test.st

2. Scripts can still be urls
#squeak -- http://installer.pbwiki.com/f/test.st

3. Scripts can invoke specific functionality of classes which implement
launchFrom:

e.g.

#squeak -headless -- MCWorkingCopy report=workingCopies +quit

#squeak -- SystemNavigation report=printCategoriesAndClasses to="*.txt"
+quit

#squeak -headless -- Script print="2+2"

#squeak -headless -- Script eval="stdout nextPutAll: 'hello world'; cr."

on an image with seaside

#squeak -headless -- WAKom start=8080

I havent tried this with a KernelImage yet..

enjoy

Keith

 

       
       
               
___________________________________________________________
All new Yahoo! Mail "The new Interface is stunning in its simplicity and ease of use." - PC Magazine
http://uk.docs.yahoo.com/nowyoucan.html

On Wed, 14 Feb 2007 21:53:44 -0800, Keith Hodges  
<[hidden email]> wrote:

> This has been tested on linux, but it looks like some help is needed to  
> get it working on mac osx.

Has anyone worked it out in Windows?

Blake wrote:
> On Wed, 14 Feb 2007 21:53:44 -0800, Keith Hodges
> <[hidden email]> wrote:
>
>> This has been tested on linux, but it looks like some help is needed
>> to get it working on mac osx.
>
> Has anyone worked it out in Windows?
>
>
Unfortunately my Windows laptop is screenless at the moment.

Keith

               
___________________________________________________________
Inbox full of spam? Get leading spam protection and 1GB storage with All New Yahoo! Mail. http://uk.docs.yahoo.com/nowyoucan.html

Blake wrote:
> On Wed, 14 Feb 2007 21:53:44 -0800, Keith Hodges
> <[hidden email]> wrote:
>
>> This has been tested on linux, but it looks like some help is needed
>> to get it working on mac osx.
>
> Has anyone worked it out in Windows?
>
As far as I am aware most of the squeak side of things should work from
Window tos. I did try it a while ago.

The main difference that I discovered was that the unix vm does not
require the '.image' for your image name, whereas the windows vm does.

At the moment the wiring for stdin and stdout is not likely to be
supported on windows, so the Script print="2+2" example is unlikely to
work. But as they say where there is a will there is a way.

;-)

Keith

               
___________________________________________________________
The all-new Yahoo! Mail goes wherever you go - free your email address from your Internet provider. http://uk.docs.yahoo.com/nowyoucan.html