ImageFormat updates

 
Hi,

Is there an Inbox to upload patches to VMMaker for review?

I generated a small fix to ImageFormat to generate magic pattern checks
at offset 512 also. I think some old Etoys images use this offset with a
launch script stuffed in the first 512 bytes. Is this deprecated now?

I also feel 'unix' should be dropped from the method's name. file(1)
utility originated in Unix but is related to file contents and not to
kernel. It should be available for Mac/Win too.

I have also attached the resulting magic file and a workspace script to
generate a bunch of test header files to test the magic file. e.g.

  $ file -m magic *.image

Regards .. Subbu

 

 
Hi Subbu,

On Sun, Apr 30, 2017 at 10:26:18PM +0530, K K Subbu wrote:
>  
> Hi,
>
> Is there an Inbox to upload patches to VMMaker for review?

That's probably a good idea, although posting to the vm-dev list as you
are doing here is fine for ImageFormat.

>
> I generated a small fix to ImageFormat to generate magic pattern checks
> at offset 512 also. I think some old Etoys images use this offset with a
> launch script stuffed in the first 512 bytes. Is this deprecated now?

The 512 byte offset is a trick for letting the image file work as a
shell script, using a shebang line to run the image. I don't think it
gets used very often, but hopefully it still works, and that is the reason
for the offset check.

I don't know if it shsould go into the magic file entries, because when
set up in this manner it actually would be an executeble script. I guess
you could look at it either way though, either as a shell script or as
a Smalltalk image file.

>
> I also feel 'unix' should be dropped from the method's name. file(1)
> utility originated in Unix but is related to file contents and not to
> kernel. It should be available for Mac/Win too.

As far as I know it is a unix utility. It would certainly be available
on OS X (which is a unix), though I'm not sure of its status on Windows.

>
> I have also attached the resulting magic file and a workspace script to
> generate a bunch of test header files to test the magic file. e.g.
>
>  $ file -m magic *.image
>
> Regards .. Subbu
>

Thanks! I'll take a look at it.

Dave

 
On Sun, Apr 30, 2017 at 02:12:27PM -0700, Eliot Miranda wrote:
Oops, the problem is in the ImageFormat class>>simpleName that was added
in our last update to imageFormat. The requiresClosuresSupport attibute
of the image does not imply cog/squeak, so we will need to fix that or
revert the change.

Dave

 
On Monday 01 May 2017 07:01 AM, David T. Lewis wrote:

I am confused. simpleName is intended as a readable name for the magic
code. Does the above mean 64-bit images will never have closure bit set
or that closure bit is always implied for 64b and will be handled by
squeak64 automatically?

# classic, +closure, +closure+spur

32-bit = squeak, cog (+C), spur (+C+spur objmem)
64-bit = squeak64?, squeak64? (+C?), spur64 (+C+spur objmem)

Regards .. Subbu

 
On Sat, May 06, 2017 at 06:16:12PM +0530, K K Subbu wrote:
Hi Subbu,

The first 64-bit Squeak image was a 64-bit V3 image with no closure support
that was made by Dan Ingalls and Ian Piumarta in 2005. This is image format
68000.

That original file is still available at http://squeakvm.org/squeak64/dist3/.
At the time of its original release, the VM was still saving the image as
if it was a 6502 image, so that is the magic number in the original file.
But the VM was soon fixed, and the original image was re-saved with the
correct file header information, which is image format 68000.

After Eliot added full closure support to Squeak, the image format for the
64-bit V3 image became 68002. In other words, if you take a 32-bit V3
closure enabled image and use SystemTracer to convert it to 64-bits, the
resulting image is in format 68002.

When Cog was introduced, the image format changed to allow Float objects
(which require two 32-bit words of space in the image) to be stored in
native word order. This is image format 6505 for 32-bit V3 image. If that
image was to be converted to a 64-bit V3 image, the image format would
be 68003. However, no Cog VM has been developed for this, so no 68003
images currently exist. Note, the float word ordering is a relatively
minor change in the image itself, and interpreter VMs simply load 6505
images by flipping the word order back. This means that a 32-bit 6505
image can be converted to 64-bit V3 by saving it with an interpreter VM,
then tracing it to 68002 format.

The 64-bit images that we use today are Spur images, and their format
number is currently 68021. These run with Cog/Spur VMs, and cannot be
loaded by an interpreter VM.

In general, an image in any given format does not require a specific VM.
For example, a 32-bit V3 image in format 6505 can be run with a Cog VM,
and interpreter VM, a SqueakJS VM, and probably a couple others. Thus,
even though format 6505 was introduced to support Cog (and stack interpreter)
VMs, it is not a "cog image", and that is why the ImageFormat utility
describes it as "a 32-bit image with closure support and float words
stored in native platform order".

ImageFormat provides a summary of the image formats as follows:

  "ImageFormat versionDescriptions do: [:e | Transcript cr; show: e]"

  a 64-bit image with no closure support and no native platform float word order requirement (68000)
  a 64-bit image with closure support and no native platform float word order requirement (68002)
  a 64-bit image with closure support and float words stored in native platform order (68003)
  a 64-bit image with closure support and float words stored in native platform order using Spur object format (obsolete) (68019)
  a 64-bit image with closure support and float words stored in native platform order using Spur object format (68021)
  a 32-bit image with no closure support and no native platform float word order requirement (6502)
  a 32-bit image with closure support and no native platform float word order requirement (6504)
  a 32-bit image with closure support and float words stored in native platform order (6505)
  a 32-bit image with closure support and float words stored in native platform order using Spur object format (6521)


Dave

 
On Saturday 06 May 2017 09:33 PM, David T. Lewis wrote:
> The 64-bit images that we use today are Spur images, and their format
> number is currently 68021. These run with Cog/Spur VMs, and cannot be
> loaded by an interpreter VM.

Thank you, Dave, for your patient explanation. I am attaching a CS where
I just use three names squeak/squeak64/spur64 for the images. The
corresponding magic file, where capability bits are shown as flags, is
also attached.

I am also attaching a changeset for printDescription:on: where I have
abbreviated your explanation as a comment.

BTW, shouldn't there be a class method in ImageFormat which captures
obsolete or deprecated magic codes? It will save a lot of confusion down
the road.

Regards .. Subbu