Re: [Vm-dev] search order for plugins

On Friday 27 April 2007 1:58 am, John M McIntosh wrote:
> I'll note the Mac carbon VM has a info.plist flag that is turned on
> to ensure the tryLoading logic tries less hard and considers the
> internall bundle's application folder first so
> the the number of opens is *considerably* less.
True. But this is not portable. Does the order

  -plugin overrides followed by dirname(argv[0])

have the same effect but in a portable way?

Regards .. Subbu

On Friday 27 April 2007 2:03 am, Andreas Raab wrote:
> subbukk wrote:
> > An strace of squeak (3.7.7) startup reveals 181 attempts to open of which
> > 179 fails with ENOENT.
>
> Why is that a problem?
The primary risk is that we may hit a file which may not be what we want. E.g.
If I happen to run a test squeak vm from /usr/lib/squeak/3.7-7, a wrong
plugin would be picked.

A secondary problem is that each open in an unlikely location costs time and
processor cycles. This may not matter on desktops but will impact small
factor machines like XO (unnecessary battery drain).

A second

BTW, running (Kubuntu Dapper)

 squeak -plugins /usr/lib/squeak/3.7-7 -vm-sound-OSS squeak.image

reduced the number of opens for vm-* plugins from 181 to 2 and the number of
failures to 0. On Unix, the search order for sound modules is MACOS, Sun and
only then OSS :-(. We could compile each platform's binary to be smart in
picking the most likely display and sound plugins.

Regards .. Subbu

On Friday 27 April 2007 3:11 pm, Bert Freudenberg wrote:
> Ian, I did not see your reply on-list ... which may explain why you
> appear "silent" at times ;)
>
> Maybe you're posting with the wrong address?
Ian's reply came to me direct. I assumed he must have replied to me direct
instead of to the list by mistake and posted my reply to this list.

Oooops! .. Subbu

On Friday 27 April 2007 8:18 pm, Ian Piumarta wrote:
> On Apr 26, 2007, at 11:53 PM, subbukk wrote:
> > There is the command line override to take care of special cases.
>
> Which just sets the same internal variable as is set by
> SQUEAK_PLUGIN_PATH.
squeakPlugins is set from SQUEAK_PLUGINS and overridden by -plugins, while
SQUEAK_PLUGIN_PATH is tried after that. I am on svn 1638. Perhaps the code is
still in flux?

> >    libpath = -plugin option if given followed by dirname of argv[0]
>
> There is no guarantee that argv[0] contains anything beyond the word
> 'squeak'.
I am not sure about MAC, but Unix specs require execv() calls to pass
pathnames in argv[0]. Bert pointed out that this could be a link (hard or
sym). But composed properly, vm will not need any link.

A little bit of background behind my suggestions:

On my Kubuntu system, Squeak (v3.7-7) has two launcher scripts - startsqueak
and npsqueakrun - and an annoying dependency on zenity (I use KDE). In
hacking together an unified launcher, I noticed vm startup code used a lot of
logic that were better handled in launcher where portability errors can
easily be fixed. The vm shouldnt have to guess and poke around to find files.
The attached launcher, still a work in progress, works both on console and in
X11, plays nicely with both GNOME and KDE.

Being new to vm code, I could be way off the mark.

Regards .. Subbu

On Apr 27, 2007, at 12:25 PM, subbukk wrote:

> squeakPlugins is set from SQUEAK_PLUGINS and overridden by -
> plugins, while
> SQUEAK_PLUGIN_PATH is tried after that. I am on svn 1638. Perhaps  
> the code is
> still in flux?

Ahh, sorry for my confusion.  These then should probably be the same  
thing and support multiple search locations and replace the initial  
'./'.

> I am not sure about MAC, but Unix specs require execv() calls to pass
> pathnames in argv[0].

Please post a ref to the part of the POSIX, SUS, X/Open, or whatever  
standard you like, specification that says this.

> On my Kubuntu system

You have a completely different set of issues, tradeoffs, assumptions  
and environmental characteristics compared with <insert name of any  
non GNU/Linux-based Unix here>.

> and an annoying dependency on zenity (I use KDE).

That's not the fault of the VM, it's the fault of whoever rebuilt and  
repackaged it.

I'll combine the plugin stuff and change the plugin search behaviour  
to be more rational.  If your Linux launcher ever ends up needing  
essential platform VM support changes that do not adversely affect  
non-Linux Unix platforms, or you find some platform support that can  
be eliminated for every species of Unix equally, then let me know.

Cheers,
Ian

On Saturday 28 April 2007 1:47 am, Ian Piumarta wrote:
> On Apr 27, 2007, at 12:25 PM, subbukk wrote:
> > I am not sure about MAC, but Unix specs require execv() calls to pass
> > pathnames in argv[0].
>
> Please post a ref to the part of the POSIX, SUS, X/Open, or whatever
> standard you like, specification that says this.
http://www.opengroup.org/onlinepubs/009695399/functions/execv.html
The specific part is:
 The value in argv[0] should point to a filename that is associated with the
process being started by one of the exec functions

Some pre-98 Unix implementations (Solaris 2.5?) used to imply that directory
names *may* be left out, but I doubt if these survived past Y2K.

Regards .. Subbu

On Apr 28, 2007, at 1:10 AM, subbukk wrote:

> The specific part is:
>  The value in argv[0] should point to a filename that is associated  
> with the
> process being started by one of the exec functions

You haven't understood the standard.

The 'should point to' in the above sentence does not say 'shall point  
to'.  The former is absolutely *not* a requirement for compliance  
with the standard; the latter absolutely *is*.

Look further down the same page at their execve() example, which  
clearly contradicts what you claim.

Regardless, it's the caller that sets up argv[] for the child in  
execv*().  Bash empirically (read the source code) preserves  
precisely the command name typed by the user; it makes no attempt to  
rewrite argv[0] to include dir information inferred from searching  
PATH.  The experiment is a trivial (3-line) program and neither BSD  
nor Linux nor Darwin behave any differently than implied by the 'ls'  
example in the standard.  Dir information gathered from searching  
PATH is not passed to the child during the execve().

Cheers,
Ian

On Saturday 28 April 2007 10:31 pm, Ian Piumarta wrote:
> Regardless, it's the caller that sets up argv[] for the child in
> execv*().  Bash empirically (read the source code) preserves
> precisely the command name typed by the user;
What you say is true - for binary executables. For scripts, see below.

>> it makes no attempt to
> rewrite argv[0] to include dir information inferred from searching
> PATH....
For scripts, it does pathname expansion, opens the file to detect shebang
interpreter and then passes the fullpath as $0 to the interp. Therefore, $0
in a shell script will contain the full pathname.

E.g.:
# echo 'echo $0' >/bin/x.sh && x.sh
/bin/x.sh
# cat >/tmp/x.c <<EOF
#include <stdio.h>
#include <unistd.h>
main(int argc, char *argv[])
{
        printf("arg0 is %s\n", argv[0]);
        execlp("x.sh", "x.sh", (char *)0);
}
EOF
# cc -o /bin/x.tmp /tmp/x.c && x.tmp
arg0 is x.tmp
/bin/x.sh

Relocatable packages like autoconf and ghostscript (gs-common) have been
using 'dirname $0' in their launcher scripts for a long time now, so the
idiom is portable.

Coming back to my suggestions - we could simplify squeak vm by moving path
logic into a launcher script. Invoked with the fullpath, the vm's plugin
probe does not have to fail so many times before hitting the right location.
Secondly, we could also unify the standalone launcher and plugin launcher.

Regards .. Subbu

On Apr 30, 2007, at 3:08 AM, subbukk wrote:

> For scripts, it does pathname expansion, opens the file to detect  
> shebang
> interpreter and then passes the fullpath as $0 to the interp.  
> Therefore, $0
> in a shell script will contain the full pathname.

I've tried this on many different shells and it appears to be  
reliable only if there are no dirnames in the command.  Typing a  
relative path to the script breaks it.  However, it's nothing that a  
call to realpath() can't cure so here's what I propose to do, unless  
someone else can think of a reason not to:

1. Remove ./ from the list of places searched.
2. Merge SQUEAK_PLUGINS and SQUEAK_PLUGIN_PATH into one.
3. Fix any relative paths in SQUEAK_PLUGIN_PATH via realpath().
4. Make SQUEAK_PLUGIN_PATH be the first place searched.

This would give you what you're asking for (use the -plugins option)  
with the smallest change to the behaviour of the binary.

Cheers,
Ian

On Apr 30, 2007, at 12:17 , Ian Piumarta wrote:

Sounds good to me.

- Bert -

Ok, as mentioned I had in the macintosh carbon vm a setting where we  
don't search everywhere for a plugin, only say the SQUEAK_PLUGIN_PATH .

Then for example if we look for the built-in LargeIntegers plugin  we  
only look in SQUEAK_PLUGIN_PATH (one lookup) before using the  
internal version.
That avoids hunting in 80 *+/-* other places

On Apr 30, 2007, at 9:17 AM, Ian Piumarta wrote:

--
========================================================================
===
John M. McIntosh <[hidden email]>
Corporate Smalltalk Consulting Ltd.  http://www.smalltalkconsulting.com
========================================================================
===

On Monday 30 April 2007 9:47 pm, Ian Piumarta wrote:
> 1. Remove ./ from the list of places searched.
> 2. Merge SQUEAK_PLUGINS and SQUEAK_PLUGIN_PATH into one.
> 3. Fix any relative paths in SQUEAK_PLUGIN_PATH via realpath().
> 4. Make SQUEAK_PLUGIN_PATH be the first place searched.
>
> This would give you what you're asking for (use the -plugins option)
> with the smallest change to the behaviour of the binary.
Sounds good.

squeakPlugins starts off with VM_LIBDIR and then gets set to
SQUEAK_PLUGIN_PATH (if defined) and then reinitialized with -plugins (if
provided). Then module loaded needs to look only into squeakPlugins.

Regards .. Subbu