VM/Image magic numbers (Re: [Pharo-users] Pharo 6 snap install)

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

VM/Image magic numbers (Re: [Pharo-users] Pharo 6 snap install)

Ben Coman
 
The "magic decode for image files" seems related and possibly *very* useful....
http://forum.world.st/magic-decode-for-image-files-td4941712.html#a4941837

Now traditionally the VM has been backward compatible with old Images,
but Pharo doesn't necessarily adhere to that and policy IIUC is
release specific VM/Image pairs.
So does Pharo require extra magic numbers to co-ordinate this?
And how would this interact with OpenSmalltalk policy on these magic numbers?

cheers -ben

On Sat, Apr 15, 2017 at 4:20 AM, Stephane Ducasse
<[hidden email]> wrote:

>
> This is what we always have when we release and we freeze it.
> The vm 60 will be compatible with latest pharo 60 image.
>
>
>
> On Fri, Apr 14, 2017 at 12:09 PM, Luke Gorrie <[hidden email]> wrote:
>>
>> On 14 April 2017 at 08:53, Stephane Ducasse <[hidden email]> wrote:
>>>
>>> We always try to have a major release around mid/end of april
>>
>>
>> Once the dust settles, I think it would be wonderful to have this invariant:
>>
>> - The latest Pharo VM source release is compatible with the latest Pharo image.
>>
>> This would make it easy for me to help nix users. My job will be to package the latest source release of the VM, period, and I will know that users can use this to run the latest image. I don't have much scope to screw things up and make people sad e.g. by shipping a bad commit from the master branch.
>>
>> (Of course it makes life harder for you upstream Pharo maintainers. Now updating the VM source release is a blocker for releasing a new image. Could be that this is too constraining in practice, I dunno. I'm just a guy who wants to setup a stable Pharo that I can use to build an application :-))
>>
>>
>
Reply | Threaded
Open this post in threaded view
|

Re: [Pharo-users] VM/Image magic numbers (Re: Pharo 6 snap install)

Luke Gorrie
 
On 15 April 2017 at 01:22, Ben Coman <[hidden email]> wrote:
The "magic decode for image files" seems related and possibly *very* useful....
http://forum.world.st/magic-decode-for-image-files-td4941712.html#a4941837

Now traditionally the VM has been backward compatible with old Images,
but Pharo doesn't necessarily adhere to that and policy IIUC is
release specific VM/Image pairs.
So does Pharo require extra magic numbers to co-ordinate this?
And how would this interact with OpenSmalltalk policy on these magic numbers?

I would really like a copy of the script that checks the magic number and starts the right VM. David, could you share yours please?

I started writing

case file -m @share@/magic "$image" in
    'Smalltalk image V3 32b*')
        vm=@pharo-vm-cog@
        ;;
    'Smalltalk image Spur 32b*')
        vm=@pharo-vm-spur@
        ;;
    'Smalltalk image Spur 64b*')
        vm=@pharo-vm-spur64@
        ;;
    *)
        echo unrecognized image file format
        ;;
esac

... but would prefer to avoid reinventing the wheel e.g. on how to find the image name in amongst the various options passed to $@.

I would quite like for my package to be able to open the pharo-launcher image and the various images that this downloads. This means supporting pre-spur images. I would ideally like to do that by building a non-spur version of the latest VM, but I am not sure if that is supported for pharo, there is no mvm script for non-spur. The alternative would be to just build an old VM release with cmake.


Reply | Threaded
Open this post in threaded view
|

Re: [Pharo-users] VM/Image magic numbers (Re: Pharo 6 snap install)

David T. Lewis
 
On Tue, Apr 18, 2017 at 11:11:07PM +0200, Luke Gorrie wrote:

>  
> On 15 April 2017 at 01:22, Ben Coman <[hidden email]> wrote:
>
> > The "magic decode for image files" seems related and possibly *very*
> > useful....
> > http://forum.world.st/magic-decode-for-image-files-td4941712.html#a4941837
> >
> > Now traditionally the VM has been backward compatible with old Images,
> > but Pharo doesn't necessarily adhere to that and policy IIUC is
> > release specific VM/Image pairs.
> > So does Pharo require extra magic numbers to co-ordinate this?
> > And how would this interact with OpenSmalltalk policy on these magic
> > numbers?
> >
>
> I would really like a copy of the script that checks the magic number and
> starts the right VM. David, could you share yours please?
Attached.

Dave

Reply | Threaded
Open this post in threaded view
|

Re: [Pharo-users] VM/Image magic numbers (Re: Pharo 6 snap install)

K K Subbu
In reply to this post by Luke Gorrie
 
On Wednesday 19 April 2017 02:41 AM, Luke Gorrie wrote:

> I would really like a copy of the script that checks the magic number
> and starts the right VM. David, could you share yours please?
Luke,

You can use the binfmt_misc kernel module to get the kernel to launch
the right VM. binfmt_misc is already loaded on all recent linux kernels
for running qemu and python bytecodes, so you should be able to register
image patterns directly in /proc/sys/fs/binfmt_misc/register or through
update-binfmts utility.

e.g.
$ sudo update-binfmts --install spur64 /usr/bin64/spur --offset 0
--magic '\xb5\x09\x01\x00'
...
will install the interpreter spur64 running from /usr/bin64/spur to run
64-bit little-endian files (magic 68021)

$ sudo update-binfmts --remove spur64 /usr/bin64/spur

For you own testing, you can use this alias

$ alias sqmagic='od -An -N4 -t x1'
or
$ alias sqmagic64='od -An -j4 -N4 -t x1'
for 64-bit bigendian images

You may want to filter the output through "tr -d ' '" to eliminate the
spaces.

HTH .. Subbu
Reply | Threaded
Open this post in threaded view
|

Re: [Pharo-users] VM/Image magic numbers (Re: Pharo 6 snap install)

Luke Gorrie
 
Thanks for the tip, Subbu.

I am using your 'magic' file definition in a script based on David's one. The user can write 'pharo my.image' and it will automatically run a suitable VM. I have tested this now with Moose 6.1, Pharo 6.0, Pharo64 6.0, Pharo 5.0, Pharo 4.0, Pharo 3.0, and DrGeo.

Now I am hoping to make this all work with Pharo Launcher. That image isn't starting for me at the moment, for some reason, so I have to troubleshoot. Then I hope I will be able to tell PharoLauncher to run the images with my script somehow.

The binfmts trick doesn't suit me for two reasons. One is that it creates a global association but I want to be able to install many versions of the pharo VM at the same time (selected via $PATH and managed by nix.) The other is that I consider running commands as root to be an unrealistic expectation for people wanting to run a GUI application (same reason I dislike the realtime priority warning.)

Cheers!
-Luke


Reply | Threaded
Open this post in threaded view
|

Re: [Pharo-users] VM/Image magic numbers (Re: Pharo 6 snap install)

Stephan Eggermont-3
 
PharoLauncher already has some code to use different vms to open images. It should use the smalltalk code Dave wrote however

Stephan

Verstuurd vanaf mijn iPhone

> Op 19 apr. 2017 om 12:54 heeft Luke Gorrie <[hidden email]> het volgende geschreven:
>
> Thanks for the tip, Subbu.
>
> I am using your 'magic' file definition in a script based on David's one. The user can write 'pharo my.image' and it will automatically run a suitable VM. I have tested this now with Moose 6.1, Pharo 6.0, Pharo64 6.0, Pharo 5.0, Pharo 4.0, Pharo 3.0, and DrGeo.
>
> Now I am hoping to make this all work with Pharo Launcher. That image isn't starting for me at the moment, for some reason, so I have to troubleshoot. Then I hope I will be able to tell PharoLauncher to run the images with my script somehow.
>
> The binfmts trick doesn't suit me for two reasons. One is that it creates a global association but I want to be able to install many versions of the pharo VM at the same time (selected via $PATH and managed by nix.) The other is that I consider running commands as root to be an unrealistic expectation for people wanting to run a GUI application (same reason I dislike the realtime priority warning.)
>
> Cheers!
> -Luke
>
>

Reply | Threaded
Open this post in threaded view
|

Re: [Pharo-users] VM/Image magic numbers (Re: Pharo 6 snap install)

Luke Gorrie
 
On 19 April 2017 at 13:03, Stephan Eggermont <[hidden email]> wrote:

PharoLauncher already has some code to use different vms to open images. It should use the smalltalk code Dave wrote however

I see this now.

Gets a bit tricky when the same problem is being solved in different ways in different places.

I like having the problem solved with a unix script because this works for people who aren't using the Pharo Launcher. However, I also want the Pharo Launcher to work in the same way. So I really need a way to specify the full path to the script that Pharo Launcher should exec.

Can we add such a hook to Pharo Launcher now? That would save me some pain :).


Reply | Threaded
Open this post in threaded view
|

Re: [Pharo-users] VM/Image magic numbers (Re: Pharo 6 snap install)

Stephan Eggermont-3
 
PharoLauncher also works on Windows, where a different script would be needed...

Dave's smalltalk code for recognizing the image format works everywhere (after being able to start your first image). 

The PharoLauncher code needs to be updated anyway because it only recognizes spur and pre-spur, no 64 bit. 

Please feel free to modify the PharoLauncher code so it can work with a script and instead/in addition to a vm-path for each image format. 
Simplest would be to add a setting to a script, and modifying the start image command to use it when not empty. Then one could enter either a vm path or a script

Stephan

Verstuurd vanaf mijn iPhone

Op 19 apr. 2017 om 13:13 heeft Luke Gorrie <[hidden email]> het volgende geschreven:

On 19 April 2017 at 13:03, Stephan Eggermont <[hidden email]> wrote:

PharoLauncher already has some code to use different vms to open images. It should use the smalltalk code Dave wrote however

I see this now.

Gets a bit tricky when the same problem is being solved in different ways in different places.

I like having the problem solved with a unix script because this works for people who aren't using the Pharo Launcher. However, I also want the Pharo Launcher to work in the same way. So I really need a way to specify the full path to the script that Pharo Launcher should exec.

Can we add such a hook to Pharo Launcher now? That would save me some pain :).


Reply | Threaded
Open this post in threaded view
|

Re: [Pharo-users] VM/Image magic numbers (Re: Pharo 6 snap install)

Luke Gorrie
 
Thanks for the tips, Stephen. I didn't see Dave's smalltalk code, only shell code, so maybe I missed a better solution.

I reckon I have found a suitable local maximum for now. It'll be easy for Nix users to open any image (cog/spur/spur64) and it'll also easy to download and run a few predefined images. That's more-or-less a refresh of Damien's original nix packaging.

The next step for me is to get back to writing my own application. This packaging business is only a necessary evil towards that end :).

Cheers,
-Luke