new VM directory structure for 5.3

On 11/01/20 3:53 AM, Chris Muller wrote:
> This simplification allows easy co-existence of 32-bit with 64-bit
> versions of the same VM.
If the bin directory is created with <os>-<isa>-<optional-variant>/
    linux-x86/
    linux-x86_64/
    linux-arm-armhf/
    linux-arm64/
    win-win32/
    win-win64/
    osx-x86/

then it would be simpler and easier to ship and share multiplatform code
(e.g. shared mounts). Is it too late to make the change?

> The SqueakV50.sources file must be present in at least one of those two
> directories.

*.sources does not have any arch-specific code. It would be better to
specify the shared directory only but be lenient and look in bin*
directory if it is missing there.

Regards .. Subbu

Hi all,

In Windows you just copy one folder to the next machine and everything
works. Which btw is an advantage Squeak has over Windows software that
needs installation.
I had hoped that Linux Squeak would make one step into that direction,
but there seem to be arguments against it (I'm not qualified to argue
here).

Best regards,

Herbert

Am 11.01.2020 um 06:55 schrieb K K Subbu:

Can't say about Windows, but sure you could made easy Mac and Linux and back development if have samba in your Linux.
Steps.
Go for All in one , download, unzip.
In Mac move the All in One to Applications for no complains.
I prefer have my own flavor of Squeak so
Navigate to Applications folder
Squeak5.3alpha-19130-64bit
Control key and select Package contents
Select .image and .changes
Made MySqueak folder into Documents folder and move the files.
Via samba select your Linux box and copy this folder
Also copy the Linux folders like this and do not forget put any .sources into shared


If need I available at https://discord.gg/aVeXVkC , send mail first

Edgar
@morplenauta




On 11/01/2020, 08:49, "Herbert König" <[hidden email]> wrote:

> Hi all, In Windows you just copy one folder to the next machine and
> everything works. Which btw is an advantage Squeak has over Windows software
> that needs installation. I had hoped that Linux Squeak would make one step
> into that direction, but there seem to be arguments against it (I'm not
> qualified to argue here). Best regards, Herbert

Hi all,

In Windows you just copy one folder to the next machine and everything
works. Which btw is an advantage Squeak has over Windows software that
needs installation.
I had hoped that Linux Squeak would make one step into that direction,
but there seem to be arguments against it (I'm not qualified to argue
here).

Best regards,

Herbert

Am 11.01.2020 um 06:55 schrieb K K Subbu:
> On 11/01/20 3:53 AM, Chris Muller wrote:
>> This simplification allows easy co-existence of 32-bit with 64-bit
>> versions of the same VM.
> If the bin directory is created with <os>-<isa>-<optional-variant>/
>    linux-x86/
>    linux-x86_64/
>    linux-arm-armhf/
>    linux-arm64/
>    win-win32/
>    win-win64/
>    osx-x86/
>
> then it would be simpler and easier to ship and share multiplatform
> code (e.g. shared mounts). Is it too late to make the change?
>
>> The SqueakV50.sources file must be present in at least one of those
>> two directories.
>
> *.sources does not have any arch-specific code. It would be better to
> specify the shared directory only but be lenient and look in bin*
> directory if it is missing there.
>
> Regards .. Subbu
>

I hope not.  I love this idea.  Can I suggest that for names we use the same names as the build directories minus the “build.” prefix?  linux32x86 linux64x64 linux32ARMv6 etc...

>> The SqueakV50.sources file must be present in at least one of those two directories.
>
> *.sources does not have any arch-specific code. It would be better to specify the shared directory only but be lenient and look in bin* directory if it is missing there.
>
> Regards .. Subbu
>

Simplifying is good but we need to make sure we keep things mutually up to date.

The .sources files must not be in an architecture or minor release specific directory - SqueakV5.0.sources should be in the root of the vm directory tree rather than buried down in Squeak5.3beta-19316-64bit-201912311458-Linux/bin - that would mean each and every downloaded vm update having another copy. Very wasteful of both bandwidth and disk space. Since the path for searching for the sources is in the image we need to update that method to suit whatever is chosen - I mentioned a while back that the all-in-one directory tree had this issue.

We also have expected directory paths in the ExternalSettings class(es). In ExternalSettings class>>#preferenceDirectory we refer to vmPath and then look for vmPath/prefs/{assorted preferences files}. If we want those preference files to be reasonably shared, they should likely go under the directory the sources are kept in.

Whilst looking at usage of #vmPath it seems we have a potential clash between methods like FileDirectory class>>#openSources:forImage: and SmalltalkImage>>#locateSourcesEntry

 - SmalltalkImage>>#openSourceFiles sends
 - SmalltalkImage>>#sourcesName which use #locateSourceEntry to look in the vmPath, the default directory (which may be set by the SecurityPlugin) and the 'slash' root; the found file (or one assumed in the vmPath even if it wasn't found there) is then used by
 - FileDirectory class>>#openSources:andChanges:forImage: which sends
 - FileDirectory class>>#openSources:forImage: (which does weird shit looking for a compressed file) and even weirder and dangerously *strips the carefully found full path name* and then does yet another search in several places - in this case the vmPath, the directory derived from the imagename and finally the default directory (which typically we'd anticipate is the image directory unless the SecurityPlugin has altered)

Now that lot is just plain nuts.

Just for fun there is also a useful looking FileDirectory class>>#lookInUsualPlaces: method that could likely be used to clean up a lot of this but is only sent by ServerPassword>>#serverPasswords - which actually surely ought to be integrated into the ExternalSettings stuff! Oh and GetTextTranslator class>>#setupLocaleDirs.


tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Don't sweat petty things....or pet sweaty things.

On 12/01/20 2:18 AM, Eliot Miranda wrote:
+1. It would simplify build scripts. We could have a recursive make
    $ make

which probes the host for platform settings and then invokes a recursive
make. E.g.
    $ make OS=linux32 ISA=ARM VARIANT=v6

Regards .. Subbu

On 12/01/20 8:54 AM, tim Rowledge wrote:
> Whilst looking at usage of #vmPath it seems we have a potential clash between methods like FileDirectory class>>#openSources:forImage: and SmalltalkImage>>#locateSourcesEntry
> .....
> Now that lot is just plain nuts.

There is too much coupling between Squeak and its host in the
FileDirectory classes. Squeak (vm+image) is a full-fledged virtual
machine and should be sandboxed from its underlying host.

Each plugin punches a hole through this barrier. As long as the plugin
is a consumer of host services (e.g. fonts) it is fine, but if it
involves writes/updates (like in FileDirectory) sandboxing needs to be
enforced. For a multi-platform machine like Squeak, leaking
host-specific details like pathnames or keyboard codes into the image
should be a big no no.

Earlier, I had proposed that we use symbolic tokens between Squeak and
its plugins so that host-specific details don't leak into Squeak. We
could use bookmarks like #vm #sources #image or #prefs etc and let the
FileDirectory plugin map it to the exact paths.

Regards .. Subbu

> On 2020-01-12, at 2:04 AM, K K Subbu <[hidden email]> wrote:
>
> There is too much coupling between Squeak and its host in the FileDirectory classes. Squeak (vm+image) is a full-fledged virtual machine and should be sandboxed from its underlying host.
>
> Each plugin punches a hole through this barrier. As long as the plugin is a consumer of host services (e.g. fonts) it is fine, but if it involves writes/updates (like in FileDirectory) sandboxing needs to be enforced. For a multi-platform machine like Squeak, leaking host-specific details like pathnames or keyboard codes into the image should be a big no no.

I think I disagree with that. There are certainly cases where you want to have some variety of sandbox around your system - I guess a deployed application may be an example. And for all its probable faults, that is what the SecurityPlugin is an attempt to provide.

But for my development tool I want total access.

For handling the host specific matters I'd prefer as much as possible for the vm to *not* deal with them but to have system specific *image code*. It isn't always practical, certainly, but having code to convert OS specific keyboard info to what we want is much, much, more flexible than sticking a lot in the vm where we can't fix issues so quickly.  

For file access I'd much prefer the image code to call out to host apis via ffi/alien/systemcall than have a complex of not terribly flexible plugins wrapping it up opaquely. Eliot & I did that decades ago and it was very useful. Of course, there are places where it's simply too painful due to complex OS dances that involve calling back callback interrupt handler handlers that handle callbacks by interrupting... something.

In fact, trying to improve that opacity was why plugins got invented in the first place; Andreas & I wanted a way to wrap vm code in a way that meant we could update *parts* of the vm rather than all of it at once. That's why external plugins (are supposed to) over-ride internal ones. And of course that is a weakness of the SecurityPlugin concept, since I can simply put a version that skips over any checks into the appropriate directory and end-run the checks.

Oh and {insert generic long term whine about awful state of filename/directory stuff} whine, whimper.

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Strange Opcodes: FART: Fill Accumulator from Result if True