Smalltalk › Squeak › Squeak - Dev

Spur 64-bits. Ugh, this could be a slog...

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

7 messages Options

Eliot Miranda-2

Spur 64-bits. Ugh, this could be a slog...

Hi All,

I'm making good progress on 64-bit Spur in the Stack VM simulator. But I've just noticed an image-level issue which could be indicative of lots of 32-bit assumptions baked into the Squeal/Pharo/Newspeak systems.

SmallInteger>>digitAt: n

"Answer the value of an indexable field in the receiver. LargePositiveInteger uses bytes of base two number, and each is a 'digit' base 256. Fail if the argument (the index) is not an Integer or is out of bounds."

n>4 ifTrue: [^ 0].

self < 0

ifTrue:

[self = SmallInteger minVal ifTrue:

["Can't negate minVal -- treat specially"

^ #(0 0 0 64) at: n].

^ ((0-self) bitShift: (1-n)*8) bitAnd: 16rFF]

ifFalse: [^ (self bitShift: (1-n)*8) bitAnd: 16rFF]

This assumes that SmallInteger is only ever 4 bytes, which is unacceptably wasteful for my approach to 64-bits. In 64-bit Spur, SmallIntegers are 61-bit 2's complement.

I'm raising this example at this point to see if the community might find similar issues and bring them to my attention.

best,

Eliot

David T. Lewis

Re: [Vm-dev] Spur 64-bits. Ugh, this could be a slog...

On Tue, Nov 18, 2014 at 07:12:12PM -0800, Eliot Miranda wrote:

>
> Hi All,
>
> I'm making good progress on 64-bit Spur in the Stack VM simulator. But
> I've just noticed an image-level issue which could be indicative of lots of
> 32-bit assumptions baked into the Squeal/Pharo/Newspeak systems.
>
> SmallInteger>>digitAt: n
> "Answer the value of an indexable field in the receiver.
> LargePositiveInteger uses bytes of base two number, and each is a 'digit'
> base 256. Fail if the argument (the index) is not an Integer or is out of
> bounds."
> n>4 ifTrue: [^ 0].
> self < 0
> ifTrue:
> [self = SmallInteger minVal ifTrue:
> ["Can't negate minVal -- treat specially"
> ^ #(0 0 0 64) at: n].
> ^ ((0-self) bitShift: (1-n)*8) bitAnd: 16rFF]
> ifFalse: [^ (self bitShift: (1-n)*8) bitAnd: 16rFF]
>
> This assumes that SmallInteger is only ever 4 bytes, which is unacceptably
> wasteful for my approach to 64-bits. In 64-bit Spur, SmallIntegers are
> 61-bit 2's complement.
>
> I'm raising this example at this point to see if the community might find
> similar issues and bring them to my attention.

There will be quite a few issues like this scattered throughout the image.

My suggestion would be to focus first on getting the 64-bit Spur image
running with no changes to SmallInteger minVal and maxVal. Then, as a
separate follow up step, expand the range of SmallInteger in the image
and work out all the changes required to support it.

Another possible way to attack the problem is to start with a Squeak
trunk image in 68002 image format (64-bit image), and work out the
issues associated with extending SmallInteger minVal and maxVal to
take advantage of the larger immediate integers range. This could be
independent of the work you are doing now, so if someone else wanted
to work on the "expand range of SmallInteger" aspect of the project,
I expect that this can be done in the existing 68002 image, such that
the resulting changes would be directly applicable to the format 68019
(64-bit) Spur images.

Dave

David T. Lewis

Re: [Vm-dev] Re: Spur 64-bits. Ugh, this could be a slog...

In reply to this post by Eliot Miranda-2

On Tue, Nov 18, 2014 at 07:59:30PM -0800, Ryan Macnak wrote:
>
> What about bytecode pc's? These are indexed based on the physical size of
> the literals. I don't like the idea of debugging pc -> source mappings, pc
> coverage data, etc needing invalidation after starting an image on a
> platform with a different word size.

I may be misunderstanding your question, but I do not think that this will
be an issue. All byte addressible objects need to work properly in any image,
including 32 and 64 bit images. If that part does not work, then the image
will not run, so no problem ;-)

With respect to platform word size (e.g. 64 bit OS versus 32 bit OS), the
images should all run bit-identically regardless of platform word size. A
64-bit image can (and does) run on a 32-bit host platform, given a suitably
compiled VM. And a 32-bit image will run on a 64-bit host platform, given a
suitably compiled VM (or more commonly, a 32-bit VM with 32-bit compatibility
libraries installed on the platform).

Dave

>
> On Tue, Nov 18, 2014 at 7:12 PM, Eliot Miranda <[hidden email]>
> wrote:
>
> > Hi All,
> >
> > I'm making good progress on 64-bit Spur in the Stack VM simulator.
> > But I've just noticed an image-level issue which could be indicative of
> > lots of 32-bit assumptions baked into the Squeal/Pharo/Newspeak systems.
> >
> > SmallInteger>>digitAt: n
> > "Answer the value of an indexable field in the receiver.
> > LargePositiveInteger uses bytes of base two number, and each is a 'digit'
> > base 256. Fail if the argument (the index) is not an Integer or is out of
> > bounds."
> > n>4 ifTrue: [^ 0].
> > self < 0
> > ifTrue:
> > [self = SmallInteger minVal ifTrue:
> > ["Can't negate minVal -- treat specially"
> > ^ #(0 0 0 64) at: n].
> > ^ ((0-self) bitShift: (1-n)*8) bitAnd: 16rFF]
> > ifFalse: [^ (self bitShift: (1-n)*8) bitAnd: 16rFF]
> >
> > This assumes that SmallInteger is only ever 4 bytes, which is unacceptably
> > wasteful for my approach to 64-bits. In 64-bit Spur, SmallIntegers are
> > 61-bit 2's complement.
> >
> > I'm raising this example at this point to see if the community might find
> > similar issues and bring them to my attention.
> > --
> > best,
> > Eliot
> >

Eliot Miranda-2

Re: Spur 64-bits. Ugh, this could be a slog...

In reply to this post by Eliot Miranda-2

Hi Ryan,

On Tue, Nov 18, 2014 at 7:59 PM, Ryan Macnak <[hidden email]> wrote:

What about bytecode pc's? These are indexed based on the physical size of the literals. I don't like the idea of debugging pc -> source mappings, pc coverage data, etc needing invalidation after starting an image on a platform with a different word size.

I think this is taken care of. For example, CompiledMethod>>initialPC already consults wordSize.

CompiledMethod>>initialPC

"Answer the program counter for the receiver's first bytecode."

^ (self numLiterals + 1) * Smalltalk wordSize + 1

and of course the image bootstrap maps pcs already. i.e.

Spur32to64BitBootstrap>>fillInPointerObjectWithPC: obj64 from: obj32

| method |

self fillInPointerObject: obj64 from: obj32.

(heap64 classIndexOf: obj64) = ClassBlockClosureCompactIndex ifTrue:

[method := heap32

fetchPointer: MethodIndex

ofObject: (heap32

fetchPointer: ClosureOuterContextIndex

ofObject: obj32).

self incrementPCField: ClosureStartPCIndex ofObject: obj64 for: method].

(heap64 classIndexOf: obj64) = ClassMethodContextCompactIndex ifTrue:

[method := heap32

fetchPointer: MethodIndex

ofObject: obj32.

self incrementPCField: InstructionPointerIndex ofObject: obj64 for: method]

Spur32to64BitBootstrap>>incrementPCField: fieldIndex ofObject: obj64 for: method32

| value nLits |

value := heap64 fetchPointer: fieldIndex ofObject: obj64.

(heap64 isIntegerObject: value)

ifTrue:

[nLits := heap32 literalCountOf: method32.

heap64

storePointerUnchecked: fieldIndex

ofObject: obj64

withValue: (heap64 integerObjectOf: nLits + LiteralStart * 4 + (heap64 integerValueOf: value))]

ifFalse:

[self assert: (reverseMap at: value) = heap32 nilObject]

So I'm more looking for issues where the code implicitly assumes 32-bit pointers, such as digitAt: below.

On Tue, Nov 18, 2014 at 7:12 PM, Eliot Miranda <[hidden email]> wrote:
Hi All,

I'm making good progress on 64-bit Spur in the Stack VM simulator. But I've just noticed an image-level issue which could be indicative of lots of 32-bit assumptions baked into the Squeal/Pharo/Newspeak systems.

SmallInteger>>digitAt: n
"Answer the value of an indexable field in the receiver. LargePositiveInteger uses bytes of base two number, and each is a 'digit' base 256. Fail if the argument (the index) is not an Integer or is out of bounds."
n>4 ifTrue: [^ 0].
self < 0
ifTrue:
[self = SmallInteger minVal ifTrue:
["Can't negate minVal -- treat specially"
^ #(0 0 0 64) at: n].
^ ((0-self) bitShift: (1-n)*8) bitAnd: 16rFF]
ifFalse: [^ (self bitShift: (1-n)*8) bitAnd: 16rFF]

This assumes that SmallInteger is only ever 4 bytes, which is unacceptably wasteful for my approach to 64-bits. In 64-bit Spur, SmallIntegers are 61-bit 2's complement.

I'm raising this example at this point to see if the community might find similar issues and bring them to my attention.
--
best,
Eliot

best,

Eliot

Eliot Miranda-2

Re: [Vm-dev] Re: Spur 64-bits. Ugh, this could be a slog...

In reply to this post by David T. Lewis

On Tue, Nov 18, 2014 at 10:03 PM, Ryan Macnak <[hidden email]> wrote:

Ah, that's true if images have a fixed word size. I had imagined that the VM would convert an image to the host word size at startup, just as Cog converts Floats to the host representation. The Float change is transparent to Smalltalk, but a word size change would be visible through the identities of objects that are either immediate or heap depending on the word size. And for mixed format objects like CompiledMethods, the offset for basicAt: for the beginning of the byte data would change.

Right. And for this reason I find it better to make the change a one-off bootstrap, even if it rewrites a 32-nit image to a 64-bit image, rather thna builds a fresh system from startup (an approach others are pursuing). It doesn't make sense burdening the VM with conversion code that is rarely used. The one-off conversion results in a more performant, reliable solution IMO, much like the V3 to Spur bootstrap itself.

On Tue, Nov 18, 2014 at 9:07 PM, David T. Lewis <[hidden email]> wrote:

On Tue, Nov 18, 2014 at 07:59:30PM -0800, Ryan Macnak wrote:
>
> What about bytecode pc's? These are indexed based on the physical size of
> the literals. I don't like the idea of debugging pc -> source mappings, pc
> coverage data, etc needing invalidation after starting an image on a
> platform with a different word size.

I may be misunderstanding your question, but I do not think that this will
be an issue. All byte addressible objects need to work properly in any image,
including 32 and 64 bit images. If that part does not work, then the image
will not run, so no problem ;-)

With respect to platform word size (e.g. 64 bit OS versus 32 bit OS), the
images should all run bit-identically regardless of platform word size. A
64-bit image can (and does) run on a 32-bit host platform, given a suitably
compiled VM. And a 32-bit image will run on a 64-bit host platform, given a
suitably compiled VM (or more commonly, a 32-bit VM with 32-bit compatibility
libraries installed on the platform).

Dave

>
> On Tue, Nov 18, 2014 at 7:12 PM, Eliot Miranda <[hidden email]>
> wrote:
>
> > Hi All,
> >
> > I'm making good progress on 64-bit Spur in the Stack VM simulator.
> > But I've just noticed an image-level issue which could be indicative of
> > lots of 32-bit assumptions baked into the Squeal/Pharo/Newspeak systems.
> >
> > SmallInteger>>digitAt: n
> > "Answer the value of an indexable field in the receiver.
> > LargePositiveInteger uses bytes of base two number, and each is a 'digit'
> > base 256. Fail if the argument (the index) is not an Integer or is out of
> > bounds."
> > n>4 ifTrue: [^ 0].
> > self < 0
> > ifTrue:
> > [self = SmallInteger minVal ifTrue:
> > ["Can't negate minVal -- treat specially"
> > ^ #(0 0 0 64) at: n].
> > ^ ((0-self) bitShift: (1-n)*8) bitAnd: 16rFF]
> > ifFalse: [^ (self bitShift: (1-n)*8) bitAnd: 16rFF]
> >
> > This assumes that SmallInteger is only ever 4 bytes, which is unacceptably
> > wasteful for my approach to 64-bits. In 64-bit Spur, SmallIntegers are
> > 61-bit 2's complement.
> >
> > I'm raising this example at this point to see if the community might find
> > similar issues and bring them to my attention.
> > --
> > best,
> > Eliot
> >

best,

Eliot

Eliot Miranda-2

Re: [Vm-dev] Spur 64-bits. Ugh, this could be a slog...

In reply to this post by David T. Lewis

On Tue, Nov 18, 2014 at 8:22 PM, David T. Lewis <[hidden email]> wrote:

On Tue, Nov 18, 2014 at 07:12:12PM -0800, Eliot Miranda wrote:
>
> Hi All,
>
> I'm making good progress on 64-bit Spur in the Stack VM simulator. But
> I've just noticed an image-level issue which could be indicative of lots of
> 32-bit assumptions baked into the Squeal/Pharo/Newspeak systems.
>
> SmallInteger>>digitAt: n
> "Answer the value of an indexable field in the receiver.
> LargePositiveInteger uses bytes of base two number, and each is a 'digit'
> base 256. Fail if the argument (the index) is not an Integer or is out of
> bounds."
> n>4 ifTrue: [^ 0].
> self < 0
> ifTrue:
> [self = SmallInteger minVal ifTrue:
> ["Can't negate minVal -- treat specially"
> ^ #(0 0 0 64) at: n].
> ^ ((0-self) bitShift: (1-n)*8) bitAnd: 16rFF]
> ifFalse: [^ (self bitShift: (1-n)*8) bitAnd: 16rFF]
>
> This assumes that SmallInteger is only ever 4 bytes, which is unacceptably
> wasteful for my approach to 64-bits. In 64-bit Spur, SmallIntegers are
> 61-bit 2's complement.
>
> I'm raising this example at this point to see if the community might find
> similar issues and bring them to my attention.

There will be quite a few issues like this scattered throughout the image.

My suggestion would be to focus first on getting the 64-bit Spur image
running with no changes to SmallInteger minVal and maxVal. Then, as a
separate follow up step, expand the range of SmallInteger in the image
and work out all the changes required to support it.

That's just a way of adding to the work. Going whole hog means less work after all. It is also really important to me to get a full 64-bit system working asap. I'm not interested in a hybrid like the existing 64-bit VM. What I am doing though is holding off on the 61-bit Float to SmallFloat conversion. I have to write new primitives for that and that can wait.

Another possible way to attack the problem is to start with a Squeak
trunk image in 68002 image format (64-bit image), and work out the
issues associated with extending SmallInteger minVal and maxVal to
take advantage of the larger immediate integers range. This could be
independent of the work you are doing now, so if someone else wanted
to work on the "expand range of SmallInteger" aspect of the project,
I expect that this can be done in the existing 68002 image, such that
the resulting changes would be directly applicable to the format 68019
(64-bit) Spur images.

The issues are rather straight-forward. Further, the bootstrap from a 32-bit Spur to a 64-bit Spur image is quite simple. Right now the bootstrap only has 18 methods. It has been by far the least complex bootstrap I've done, and I've done Smalltalk-80 V2 16-bit to BrouHaHa 32-bit, VW 32-bit to 64-bit, VW 3 to VW 5, Squeak V3 to Spur 32 bit, Squeak BlockContext to Squeak Closures.

best,

Eliot

Nicolas Cellier

Re: [Vm-dev] Spur 64-bits. Ugh, this could be a slog...

Hi Eliot, no time to check but I'm quite sure that many bit-tricks assume 31 bits wide SmallIntegers (check highBit and the like).

Nicolas

2014-11-19 19:26 GMT+01:00 Eliot Miranda <[hidden email]>:

On Tue, Nov 18, 2014 at 8:22 PM, David T. Lewis <[hidden email]> wrote:
On Tue, Nov 18, 2014 at 07:12:12PM -0800, Eliot Miranda wrote:
>
> Hi All,
>
> I'm making good progress on 64-bit Spur in the Stack VM simulator. But
> I've just noticed an image-level issue which could be indicative of lots of
> 32-bit assumptions baked into the Squeal/Pharo/Newspeak systems.
>
> SmallInteger>>digitAt: n
> "Answer the value of an indexable field in the receiver.
> LargePositiveInteger uses bytes of base two number, and each is a 'digit'
> base 256. Fail if the argument (the index) is not an Integer or is out of
> bounds."
> n>4 ifTrue: [^ 0].
> self < 0
> ifTrue:
> [self = SmallInteger minVal ifTrue:
> ["Can't negate minVal -- treat specially"
> ^ #(0 0 0 64) at: n].
> ^ ((0-self) bitShift: (1-n)*8) bitAnd: 16rFF]
> ifFalse: [^ (self bitShift: (1-n)*8) bitAnd: 16rFF]
>
> This assumes that SmallInteger is only ever 4 bytes, which is unacceptably
> wasteful for my approach to 64-bits. In 64-bit Spur, SmallIntegers are
> 61-bit 2's complement.
>
> I'm raising this example at this point to see if the community might find
> similar issues and bring them to my attention.

There will be quite a few issues like this scattered throughout the image.

My suggestion would be to focus first on getting the 64-bit Spur image
running with no changes to SmallInteger minVal and maxVal. Then, as a
separate follow up step, expand the range of SmallInteger in the image
and work out all the changes required to support it.

That's just a way of adding to the work. Going whole hog means less work after all. It is also really important to me to get a full 64-bit system working asap. I'm not interested in a hybrid like the existing 64-bit VM. What I am doing though is holding off on the 61-bit Float to SmallFloat conversion. I have to write new primitives for that and that can wait.

Another possible way to attack the problem is to start with a Squeak
trunk image in 68002 image format (64-bit image), and work out the
issues associated with extending SmallInteger minVal and maxVal to
take advantage of the larger immediate integers range. This could be
independent of the work you are doing now, so if someone else wanted
to work on the "expand range of SmallInteger" aspect of the project,
I expect that this can be done in the existing 68002 image, such that
the resulting changes would be directly applicable to the format 68019
(64-bit) Spur images.

The issues are rather straight-forward. Further, the bootstrap from a 32-bit Spur to a 64-bit Spur image is quite simple. Right now the bootstrap only has 18 methods. It has been by far the least complex bootstrap I've done, and I've done Smalltalk-80 V2 16-bit to BrouHaHa 32-bit, VW 32-bit to 64-bit, VW 3 to VW 5, Squeak V3 to Spur 32 bit, Squeak BlockContext to Squeak Closures.

--
best,
Eliot