The Inbox: System-dtl.1164.mcz

David T. Lewis uploaded a new version of System to project The Inbox:
http://source.squeak.org/inbox/System-dtl.1164.mcz

==================== Summary ====================

Name: System-dtl.1164
Author: dtl
Time: 8 June 2020, 9:16:15.619687 pm
UUID: f4d402cd-389a-4600-abcb-6ba4486c07c4
Ancestors: System-mt.1163

Fix handling of the '--' command line token, and let command parameters be handled consistently by SmalltalkImage>>argumentAt:
Discussion at http://lists.squeakfoundation.org/pipermail/squeak-dev/2020-June/209945.html

=============== Diff against System-mt.1163 ===============

Item was changed:
  ----- Method: SmalltalkImage>>argumentAt: (in category 'command line') -----
  argumentAt: argumentIndex
  "Answer the i-th argument of the command line, or nil if not so many argument."
+ ^ self arguments at: argumentIndex ifAbsent: [].
+ !
- ^ self getSystemAttribute:
- argumentIndex +
- (Preferences readDocumentAtStartup
- ifTrue: [ 2 ]
- ifFalse: [ 1 ])!

Item was changed:
  ----- Method: SmalltalkImage>>arguments (in category 'command line') -----
  arguments
  "Answer an array with all the command line arguments.
  This does not include imagePath, documentPath nor any option."
 
  "Smalltalk commandLine arguments"
 
+ | args strm |
+ args := self rawArguments.
+ (args includes: '--')
+ ifTrue: [ ^args copyAfter: self imageArgumentsMarker ].
+ strm := args readStream.
+ strm atEnd ifFalse: [ strm next. "skip image name"
+ strm atEnd ifFalse: [ strm next "skip startup document name" ] ].
+ ^ strm upToEnd.
+ !
- ^Array streamContents: [:str |
- | arg i |
- i := 1.
- [i > 998 or: [(arg := self argumentAt: i) == nil]]
- whileFalse:
- [str nextPut: arg.
- i := i + 1]].!

Item was changed:
  ----- Method: SmalltalkImage>>documentPath (in category 'command line') -----
  documentPath
+ "Answer the absolute path of the document passed to the VM or nil if none."
- "Answer the absolute path of the document passed to the vm or nil if none."
  "Smalltalk commandLine documentPath"
+ ^ (self getSystemAttribute: 2)
+ ifNotNil: [ :arg | arg = self imageArgumentsMarker
+ ifTrue: [nil] ifFalse: [arg]].
+
+ !
- ^ Preferences readDocumentAtStartup ifTrue: [ self getSystemAttribute: 2 ]!

Item was added:
+ ----- Method: SmalltalkImage>>imageArgumentsMarker (in category 'command line') -----
+ imageArgumentsMarker
+ "The '--' token on the command line indicates that remaining arguments should
+ be passed to the image without interpretation, and should not be treated as e.g.
+ specification of a start script."
+
+ ^ '--'!

Item was added:
+ ----- Method: SmalltalkImage>>rawArguments (in category 'command line') -----
+ rawArguments
+ "Answer an array with all the command line arguments as delivered
+ from the VM. Any arguments for the VM itself will have been removed
+ (these are accessed with optionAt:)."
+
+ "Smalltalk commandLine rawArguments"
+
+ ^Array streamContents: [:str |
+ | arg i |
+ i := 1.
+ [i > 998 or: [(arg := self getSystemAttribute: i) == nil]]
+ whileFalse:
+ [str nextPut: arg.
+ i := i + 1]].!

On 09/06/20 1:16 am, [hidden email] wrote:
+1. I never liked this preference anyway.

Suggest splitting the arguments into image, first and rest. If first is
nil, then no further processing is required. If it is a chunk or project
file, then treat it as a startup script or project and pass the rest of
the command line to it.

E.g.
     squeak.image playmedia.st  $HOME/music/foo.mp3
     squeak.image starwars.pr $HOME/background/deathstar.png
     squeak.image repl.st 'Beeper beep!'

Regards .. Subbu

On Tue, Jun 09, 2020 at 11:46:00AM +0530, K K Subbu wrote:
That is a good idea, but I think that this kind of argument handling can
best be accomplished with additional changes in the image after we get
the basic argument passing stabilized.

At this point, I am not trying to add any new features. I just want to
get the existing argument handling to work consistent with the original
design intent, and also consistently with respect to the various ways in
which a user might enter a command line from a terminal window.

Once that is done, we can think about adding new features or different
behavior. But I'm not sure that any new features will be needed. Consider
your idea of:
     $ squeak.image playmedia.st  $HOME/music/foo.mp3

I think you will find that the playmedia.st script can directly reference
Smalltalk argumentAt 1 to find the value of $HOME/music/foo.mp3 that
was passed to the image.

Dave

The aspect of this commandline stuff that I'm keen on is being able to do the simple scripting stuff we had fun with wrt the fibonacci number benchmarks last year (or century? ) as easily as possible.

A part of that will involve making the VM not open a window until it is actually needed; which of course we have had discussions about many times.

Another part  - I think, based on recollections of looking into it back then - may involve re-ordering when image command line stuff gets treated. IIRC it was pretty much entirely based on the primarily EToys project loading methods which was very near the end of the entire image startup and probably meant far more init code than was really needed got run.  It would be nice to have a way to pass a doit to the image without having to put it in a file, for example. VW does that with a --doit "3+4" sort of expression. I'm sure serious unix aficionados would love to be able to do things like
squeak < "Benchmark new runFibonacciTest" > results.txt
as well.

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Yesterday was March 100th. March feels like the longest decade for centuries.

+1

>
> At this point, I am not trying to add any new features. I just want to
> get the existing argument handling to work consistent with the original
> design intent, and also consistently with respect to the various ways in
> which a user might enter a command line from a terminal window.

Bravo Dave!  Thanks for tackling this.  It is a PITA!!

> Once that is done, we can think about adding new features or different
> behavior. But I'm not sure that any new features will be needed. Consider
> your idea of:
>     $ squeak.image playmedia.st  $HOME/music/foo.mp3
>
> I think you will find that the playmedia.st script can directly reference
> Smalltalk argumentAt 1 to find the value of $HOME/music/foo.mp3 that
> was passed to the image.

+1.  I suspect we want something like “shift” in shell scripts to allow us to consume arguments.

Maybe arguments is an OrderedCollection in a class var, and removeFirst is used to consume arguments.  Then argument processors can arbitrarily edit the arguments, prepending or apending as desired.

>
> Dave
>

On Tue, Jun 09, 2020 at 09:37:24AM -0700, tim Rowledge wrote:
> The aspect of this commandline stuff that I'm keen on is being able to do the simple scripting stuff we had fun with wrt the fibonacci number benchmarks last year (or century? ) as easily as possible.
>
> A part of that will involve making the VM not open a window until it is actually needed; which of course we have had discussions about many times.
>
> Another part  - I think, based on recollections of looking into it back then - may involve re-ordering when image command line stuff gets treated. IIRC it was pretty much entirely based on the primarily EToys project loading methods which was very near the end of the entire image startup and probably meant far more init code than was really needed got run.  It would be nice to have a way to pass a doit to the image without having to put it in a file, for example. VW does that with a --doit "3+4" sort of expression. I'm sure serious unix aficionados would love to be able to do things like
> squeak < "Benchmark new runFibonacciTest" > results.txt
> as well.
>

Funny you should mention it, that's how I went down this rabbit hole
in the first place. I was thinking that it would be an obviously good
thing to do if we could evaluate a doIt or named script file immediately
on image startup, as opposed to waiting until all of the Etoys script
processing comes to life somewhere later in the startUp process.

But then I noticed that the command parameter passing was (still) all
messed up, so I figured it might be a good idea to fix that first.

I think that adding --doit "3+4" would be a great next step.

Dave
 

On Tue, Jun 09, 2020 at 09:42:00AM -0700, Eliot Miranda wrote:
Thanks Eliot,

I'll wait one more day and if no one yells at me, I'll move the fix to trunk.

Sending #asOrderedCollection or #readStream to Smalltalk arguments will
get the job done :-)

Dave

> On 2020-06-09, at 11:52 AM, David T. Lewis <[hidden email]> wrote:
>
> Funny you should mention it, that's how I went down this rabbit hole
> in the first place. I was thinking that it would be an obviously good
> thing to do if we could evaluate a doIt or named script file immediately
> on image startup, as opposed to waiting until all of the Etoys script
> processing comes to life somewhere later in the startUp process.

Exactly. Note that VW has several commandline switches we might want to learn from

-filein {some file names}
-doit {some strings}
-evaluate {a single string & write result to stdout & quit
-cli {start a commandline loop}


tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Useful random insult:- So dumb, he faxes face up.

Changing the subject line in case any of us remembers to actually follow
up on this good idea.

On Tue, Jun 09, 2020 at 12:06:00PM -0700, tim Rowledge wrote:
These all seem like obviously good things to do.

A possible way forward:

1) Locate Ian's GetOpt.st implementation, which according to google might
be included in GST. Otherwise just write it again from scratch.

2) Add a class as the first entry to the startUp list (maybe it would be
class GetOpt). Let the the startUp: method for that class look for arguments
in the Smalltalk arguments list that match the options in the list above
(-fileIn, -diIt, -evaluate, -cli).

3) For each of these options, provide an implementation. D'oh.

4) Proceed with the remainder of the startUp: process, leaving the original
Smalltalk arguments list unmolested.

Dave

Just for information, this is from RSqueak:

    Usage: ./rsqueak [-r|-m|-h] [-naPu] [-jpiS] [-tTslL] <path> [--] [Squeak arguments]
            <path> - image path (default: Squeak.image)
            Squeak arguments are passed on to the Squeak image rather than being processed.

          Execution:
            -r|--run <code>  - Code will be compiled and executed in
                               headless mode, result printed.
            -m|--method <selector>
                             - Selector will be sent to nil in
                               headless mode, result printed.
            -n|--num <int>   - Only with -m or -r. SmallInteger to be used as
                               receiver (default: nil).
            -a|--arg <arg>   - Only with -m. Will be used as String argument.
            -P|--process     - Only with -m or -r. Disable headless mode.
                               A high-priority Process for the new context will
                               be created. The last active Process in the image
                               will be started, but then quickly switch to the
                               synthetic high-prio Process.
            -u|--stop-ui     - Only with -m or -r. Try to stop UI-process at
                               startup. Can help benchmarking.
            --shell          - Stop after loading the image. Any code typed is
                               compiled an run.

And some others.
(Admittedly, most are a bit overly specific, but have served well, especially the shell is cool)

Best regards
        -Tobias

On Tue, Jun 09, 2020 at 09:43:56PM +0200, Tobias Pape wrote:
Thanks, also good ideas. I am guessing that this command line parsing
is done in the RSqueak VM prior to entering the image, is that right?

Dave
 

Hi

Yes, everything before -- is VM stuff.

Best regards
        -Tobias

On 09/06/20 8:03 pm, David T. Lewis wrote:
> At this point, I am not trying to add any new features. I just want to
> get the existing argument handling to work consistent with the original
> design intent, and also consistently with respect to the various ways in
> which a user might enter a command line from a terminal window.
Dave,

I wasn't trying to suggest any new feature. I got misled by the name
rawArguments. I overlooked its comments that stated that vm options are
removed.

* could this be renamed into imageArguments?
* we could use 'nil' to skip startup document.

Both these changes will sync the selector with the comments. It would
also dissuade people from putting startup code in a file named 'nil' ;-).

Regards .. Subbu

On 09/06/20 1:16 am, [hidden email] wrote:
> ----- Method: SmalltalkImage>>documentPath (in category 'command
> line') ----- documentPath + "Answer the absolute path of the
> document passed to the VM or nil if none."

This doesn't seem to be true with the unix port. It passes the document
name as given in the command line. I just tried:

  $ run images/SpurVMMaker.image ./dummy.st hello world

Here is what I see:
{SmalltalkImage current documentPath.
SmalltalkImage current vmPath.
SmalltalkImage current imagePath.
SmalltalkImage current imageName}
  #('./dummy.st'
'/opt/share/squeak/sqcogspurlinuxht/lib/squeak/5.0-201804100527/'
'/opt/share/squeak/images'
'/opt/share/squeak/images/SpurVMMaker.image')

This could create problems because the relative path for the image and
document could be different (as in this case).

If a script is used to launch the vm, the working directory of the same
must be passed through the environment for resolving relative document
path in the image.

Regards .. Subbu

On Wed, Jun 10, 2020 at 06:15:53PM +0530, K K Subbu wrote:
The SmalltalkImage>>rawArguments method should probably be private. I only
added it to shorten the SmalltalkImage>>arguments method. But if it's
confusing, we can get rid of it, or just hide it in a private category.

Dave

On Wed, Jun 10, 2020 at 07:20:09PM +0530, K K Subbu wrote:
Yes that could cause problems. But I think that this is existing behavior,
not anything related to System-dtl.1164.mcz, is that right?

If the system does not find ./dummy.st in this case, it might be necessary
to use a fully qualified path to the document file.

Thanks,
Dave

On Wed, Jun 10, 2020 at 12:04:13PM -0400, David T. Lewis wrote:
Confirming, the above is existing behavior. The document file is located
relative to the current working directory of my terminal shell, regardless
of the VM and image paths. This is also the case for a file list and other
file based operations within the image.

I'm going to go ahead and move System-dtl.1164.mcz to trunk, and we
can take that as a baseline for other argument processing concerns.

Dave
 

On Tue, Jun 09, 2020 at 10:43:27PM -0500, Chris Muller wrote:
Hi Chris,

I expect you're right about -evaluate and -cli. They have novelty
value and it's nice that they work, but I would be surprised if they
get much real use.

On the other hand, a -doit that runs right away before entering the
startup sequence might be quite helpful for broken images that are
getting wedged somewhere in the middle of startup processing.
 
Dave



> On 2020-06-09, at 11:52 AM, David T. Lewis <[hidden email]> wrote:
>
> Funny you should mention it, that's how I went down this rabbit hole
> in the first place. I was thinking that it would be an obviously good
> thing to do if we could evaluate a doIt or named script file immediately
> on image startup, as opposed to waiting until all of the Etoys script
> processing comes to life somewhere later in the startUp process.

Exactly. Note that VW has several commandline switches we might want to learn from

-filein {some file names}
-doit {some strings}
-evaluate {a single string & write result to stdout & quit
-cli {start a commandline loop}

What I most like about Dave's change is the improved separation of concerns:  it allows the OS to invoke an image with the proper startup script and/or arguments WITHOUT relying on Preferences readDocumentAtStartup being set correctly.  I can see -filein being useful for a production patch use-case, but the rest of those go back to mixing the Smalltalk environment in with the operating-system environment.  For safe and scalable headless interaction, you would just use a server.  I never knew if there was an actual use-case for -evaluate and -cli other than novelty.  Is there?

Both are useful for testing. —evaluate is useful in eg a (unix software build style) configure context.  —cli is useful for people who want to explore, learn, demo, etc.