Re: [squeak-dev] Apple starting to alert users that it will end 32-bit app support on the Mac

Hi Clément,

On Apr 12, 2018, at 8:29 AM, Clément Bera <[hidden email]> wrote:

Hi all, 

question inlined...

On Thu, Apr 12, 2018 at 4:05 PM, Eliot Miranda <[hidden email]> wrote:

 

Hi Bert,

On Apr 12, 2018, at 6:03 AM, Bert Freudenberg <[hidden email]> wrote:

If that's indeed the case we need to make a 64 bit VM for 32 bit images. I think this should be just a matter of picking the right compiler options.

How so?  AFAIA it is not.  It means, for example, changing every memory access for an instance variable from 64 to 32 bits.  It is non-trivial.

What /is/ trivial is detecting that an image's word size is unsupported on a platform and offering to convert it, a process that takes less than a minute for typical images.

 

Why does this process take a full minute ? Could we do that in 1 second ? Is it only because of SmallFloat ?

The classes affected and the reasons are

- SmallInteger <-> LargePositiveInteger, LargeNegativeInteger, since in 32-bits SmallInteger minVal & SmallInteger minVal encompass a 31-bit 2's compliment range, and in 64-bits, a 61-bit range.

- SmallFloat64 <-> BoxedFloat64, since in 32-bits there are no immediate floats and in 64-bits floats whose exponent fits in 8 bits are represented by SmallFloat64

- Context & BlockClosure since pcs must be offset by 4 bytes per literal as the CompiledCode instance's literals will be twice or half as wide

- any and all bits collections (ByteSymbol, ByteArray, DoubleWordArray et al) whose formats change because the number of odd elements (the number of elements to subtract from the product of number of slots and slot size) is different in 32 and 64 bits

So the process in going from 32 to 64 bits is

- load the 32-bit image into a 32-bit simulator

- clone all the objects that are preserved in a 64-bit simulator, creating a map

- pass through the 32-bit objects, updating the corresponding slot in the 64-bit simulator with the object at the map

- save the 64-bit image

And the process in going in the reverse direction would be to

- load the 64-bit image into a 64-bit simulator

- clone all the objects in a 32-bit simulator, creating a map

- pass through the 64-bit objects, updating the corresponding slot in the 32-bit simulator with the object at the map, or otherwise allocating a boxed object for the 64-bit immediate that has no equivalent immediate in 32-bits

- save the 32-bit image

I just timed converting a 34 Mb 32-bit Squeak 5.1 image to 64-bits on my 2.3Ghz Core i7 Mac Mini in a 63-bit Spur image.  It took 20.6 seconds to convert.  One would probably have to add another 0.5 seconds for start up costs.  And Pi is around 5x slower than Mac.  So conversion times would be anywhere from 15 seconds to 1 1/2 minutes for standard images, scaling more or less linearly with larger images.

I certainly don't see any way of doing this in orders of magnitude faster.  But that's my limitation ;-)

I mean I could dream of a world where 64-bits image under 2Gb are saved as 32 bits images to be more compact on disk. Then at start-up, the 32 bits image is converted into a 64 bits image if started on a 64 bits VM, or not if started on a 32 bits VM. We could also save image under 65k as 16 bits images, and convert them back as start-up :-). Obviously that would work only if the conversion is fast, but I think it could be with tricks to handle SmallFloats.

I don't think two passes (allocation & mapping) can be avoided because pointer objects, compiled code, and bits objects all map differently, so there's no way to compute the position (and hence the oop) of the corresponding object independently of all other objects in the heap.

Note that the code optimized by Sista is currently dependent on word size though due to overflow computation, but I guess we'll solve this problem later.

 

If we go this route we would include a headless 64-bit image that converts 32-bit images to 64-bits on 64-bit only systems, and a headless 32-bit image that converts 64-bit images to 32-bits on 32-bit only systems.

IIRC 64 to 32 bit conversion needs to be written, but I think it is trivial compared to building a 64-bit vm for 32-bit images.

Of course this applies only to Spur images.  We don't have a way of automatically converting V3 images to Spur.  Yo do that going forward we may have to depend on OS virtual machines, but somehow I can't see an ARMv8 Mac supporting x86 VMs like Parallels.  This could be the end of an interoperability era.

Otherwise we won't have any way of opening regular 32 bit images on Macs.

- Bert -

On 12 April 2018 at 14:47, Fabio Niephaus <[hidden email]> wrote:

Hi all,

According to [1], Apple has started to alert users when they open a 32-bit app on the Mac and I've seen such a dialog today myself when opening a 32-bit Squeak image.

It looks like it is time to make 64-bit the default, maybe with the next release? The Pharo community is already doing that for quite some time, so I'm wondering if there is any reason not to do the same?

Best,

Fabio

[1] https://techcrunch.com/2018/04/11/apple-starting-to-alert-users-that-it-will-end-32-bit-app-support/

--

Clément Béra
https://clementbera.github.io/

https://clementbera.wordpress.com/

> On 12-04-2018, at 9:45 AM, Eliot Miranda <[hidden email]> wrote:
>
> So the process in going from 32 to 64 bits is
> - load the 32-bit image into a 32-bit simulator
> - clone all the objects that are preserved in a 64-bit simulator, creating a map
> - pass through the 32-bit objects, updating the corresponding slot in the 64-bit simulator with the object at the map
> - save the 64-bit image
>
> And the process in going in the reverse direction would be to
> - load the 64-bit image into a 64-bit simulator
> - clone all the objects in a 32-bit simulator, creating a map
> - pass through the 64-bit objects, updating the corresponding slot in the 32-bit simulator with the object at the map, or otherwise allocating a boxed object for the 64-bit immediate that has no equivalent immediate in 32-bits
> - save the 32-bit image
>
> I just timed converting a 34 Mb 32-bit Squeak 5.1 image to 64-bits on my 2.3Ghz Core i7 Mac Mini in a 63-bit Spur image.  It took 20.6 seconds to convert.  One would probably have to add another 0.5 seconds for start up costs.  And Pi is around 5x slower than Mac.  So conversion times would be anywhere from 15 seconds to 1 1/2 minutes for standard images, scaling more or less linearly with larger images.
>
> I certainly don't see any way of doing this in orders of magnitude faster.  But that's my limitation ;-)
>
> But this computation is ideal for Scorch/Sista right?  It's a series of loops.  It only ever does the same thing because input images are extremely similar.  The image could be trained on a suitably general input set before being deployed.  It would be very interesting to see how much faster Scorch/Sista manages to make image mapping.

So I guess we would have a 'native' utility image included with the normal system, that is set up to do this conversion from 'other' to 'native' as fast as possible, then quit and start the now-native image? The VM would be unchanged for normal usage except just possibly returning an error code if a wrong format image is specified and then the startup shell script(s) could do the run-converter-image pass. Faintly possible this would be a tad quicker than a specific test-first pass. Ah, I imagine we might be clever enough to detect the wrong format in the VM and then just load the converter image anyway, since we'd want to know where it lives and no other VM setup would have been done. The only extra trick would be storing the name of the converted image to run after the processing.

Sure, on *nix it is simple enough to run a format-checker, run the converter if needed (remembering an UI to ask the user if they actually want to do that) and then run the real VM/image. Doing it under our own control seems more fun.

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
From C:\*.* to shining C:\*.*

Eliot Miranda wrote on Thu, 12 Apr 2018 14:10:38 -0700
> That's why for me by far the simplest thing is to convert.

Computers have become a hundred times faster since Squeak first came out
and while disks are also faster the difference is a lot less dramatic.