Smalltalk › Squeak › Squeak VM

Status of LargeIntergersPlugin

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

9 messages Options

David T. Lewis

Status of LargeIntergersPlugin

We have updates to LargeIntegerPlugin here:

> Name: VMMaker.oscog-eem.794
> Author: eem
> Time: 2 July 2014, 3:40:54.432 pm
> UUID: 7fd23b24-d4aa-4291-996a-760b99e0d7e2
> Ancestors: VMMaker.oscog-eem.793
>
> Fix the shift for divide issues in the LargeIntegersPlugin.
> Add code to generateDivide:on:indent: to spit out checking
> asserts if required.
>
> Change the SmartSyntaxPluginCodeGenerator to generate
> code that ifdefs out the remapOop:in: rigmarole on Spur.

We also have a more extensive set of updates that Nicolas provided in email
to the list (because saving to MC was failing at the time of the discussion).

Both sets of changes seem to work in VMM trunk, and the unit tests pass with
both implementations.

The earlier discussion on this got sidetracked by the ZipPlugin issue, but
I would like to get VMM trunk updated with whichever implementation we want
to ultimately use.

I'm not sure where we stand on this, so let me just ask:

Nicolas: Is the LargeIntegersPlugin that you provided in recent email to
vm-dev suitable for inclusion in the VMs? In other words, are you happy that
this is a final version and ready for inclusion?

Eliot: Is there any reason that we should not adopt the changes from Nicolas?
If you are comfortable using that version, can we adopt it in trunk and oscog
branches?

Thanks,
Dave

Nicolas Cellier

Re: Status of LargeIntergersPlugin

2014-07-20 20:17 GMT+02:00 David T. Lewis <[hidden email]>:

We have updates to LargeIntegerPlugin here:

> Name: VMMaker.oscog-eem.794
> Author: eem
> Time: 2 July 2014, 3:40:54.432 pm
> UUID: 7fd23b24-d4aa-4291-996a-760b99e0d7e2
> Ancestors: VMMaker.oscog-eem.793
>
> Fix the shift for divide issues in the LargeIntegersPlugin.
> Add code to generateDivide:on:indent: to spit out checking
> asserts if required.
>
> Change the SmartSyntaxPluginCodeGenerator to generate
> code that ifdefs out the remapOop:in: rigmarole on Spur.

We also have a more extensive set of updates that Nicolas provided in email
to the list (because saving to MC was failing at the time of the discussion).

Both sets of changes seem to work in VMM trunk, and the unit tests pass with
both implementations.

The earlier discussion on this got sidetracked by the ZipPlugin issue, but
I would like to get VMM trunk updated with whichever implementation we want
to ultimately use.

I'm not sure where we stand on this, so let me just ask:

Nicolas: Is the LargeIntegersPlugin that you provided in recent email to
vm-dev suitable for inclusion in the VMs? In other words, are you happy that
this is a final version and ready for inclusion?

Yes it's ready for inclusion.

Eliot: Is there any reason that we should not adopt the changes from Nicolas?
If you are comfortable using that version, can we adopt it in trunk and oscog
branches?

Thanks,
Dave

David T. Lewis

Re: Status of LargeIntergersPlugin

On Sun, Jul 20, 2014 at 09:26:18PM +0200, Nicolas Cellier wrote:

>
> 2014-07-20 20:17 GMT+02:00 David T. Lewis <[hidden email]>:
> >
> > I'm not sure where we stand on this, so let me just ask:
> >
> > Nicolas: Is the LargeIntegersPlugin that you provided in recent email to
> > vm-dev suitable for inclusion in the VMs? In other words, are you happy
> > that this is a final version and ready for inclusion?
> >
> Yes it's ready for inclusion.

Good, thank you.

> > Eliot: Is there any reason that we should not adopt the changes from
> > Nicolas? If you are comfortable using that version, can we adopt it in
> > trunk and oscog branches?

I am attaching the changes for reference.

If no objections I will add this to VMM trunk tomorrow.

Dave

LargeIntPlugin.st (29K) Download Attachment

David T. Lewis

Re: Status of LargeIntegersPlugin

Moving discussion back to vm-dev, as it may be of some general interest.

On Tue, Jul 22, 2014 at 09:16:23AM -0700, Eliot Miranda wrote:

> On Mon, Jul 21, 2014 at 5:35 PM, David T. Lewis <[hidden email]> wrote:
>
> > Hi Eliot,
> >
> > I want to check before I copy this update to the VMMaker repository - are
> > you in agreement with using Nicolas' LargeIntegerPlugin? It seems fine to
> > me, so I'll move it to VMM trunk if you agree.
> >
>
> Well, it is goodness. My concern is that it requires image changes to make
> image code still function, but haven't had time to check it out. The issue
> is that currently, large integers are in little endian order and code in
> the image accesses bytes in large integers assuming that the bytes there-in
> are in little endian order. For example see digitCompare: and digitAt:.
> Now digitAt: is implemented as simply primitive 60, the standard at:
> primitive. So on current VMs a big-endian large integer will /not/ be
> correctly accessed via digitAt:.
>
> Am I right?

I do not have a big endian machine to check this, but my assumption is that
since Squeak began on big endian machines and moved to little endian
later, and since the LargeIntegersPlugin has been around for a long time,
and since images migrated across big and little endian platforms without
any apparent problem, then we probably do not have any endianness problems
with the existing plugin.

I wrote a few tests (and added them to trunk) to validate the digit access
methods. These tests pass on my little endian PC with a Cog VM using the
old plugin, and they also pass with Nicolas' new LargeIntegersPlugin with
both the 32-bit and 64-bit image formats.

Nicolas' updated plugin seems to be fully compatible with the original
LargeIntegersPlugin, and it produces the expected results for all of the
image formats and VMs that I am able to test. I cannot confirm this for
a big endian platform, but I think that any errors on big endian box
would be exposed by the unit tests.

It looks good to me.

Nicolas, any concerns or comments?

Thanks,
Dave

timrowledge

Re: Status of LargeIntegersPlugin

> my assumption is that
> since Squeak began on big endian machines and moved to little endian
> later, and since the LargeIntegersPlugin has been around for a long time,
> and since images migrated across big and little endian platforms without
> any apparent problem, then we probably do not have any endianness problems
> with the existing plugin.

There is, or at least was, code to byteswap byte objects at image load time. Not in a good spot to check right now. I think it went
Check endian is as was
Swap all words if needed.
Swap back contents of byte objects.

Or something like that.

/tim
{insert witticism here}

David T. Lewis

Re: Status of LargeIntegersPlugin

On Tue, Jul 22, 2014 at 08:55:49PM -0700, tim Rowledge wrote:

>
> > my assumption is that
> > since Squeak began on big endian machines and moved to little endian
> > later, and since the LargeIntegersPlugin has been around for a long time,
> > and since images migrated across big and little endian platforms without
> > any apparent problem, then we probably do not have any endianness problems
> > with the existing plugin.
>
> There is, or at least was, code to byteswap byte objects at image load time. Not in a good spot to check right now. I think it went
> Check endian is as was
> Swap all words if needed.
> Swap back contents of byte objects.
>
> Or something like that.
>
> /tim

Judging by the author credited in the method comment, I suspect that you are probably right ;)

Interpreter>>reverseBytesInImage
"Byte-swap all words in memory after reading in the entire image file with bulk read.
Contributed by Tim Rowledge."

"First, byte-swap every word in the image. This fixes objects headers."
objectMemory reverseBytesFrom: objectMemory startOfMemory to: objectMemory getEndOfMemory.

"Second, return the bytes of bytes-type objects to their orginal order."
self byteSwapByteObjects.

timrowledge

Re: Status of LargeIntegersPlugin

Wholly khao. That would have to be eighteen years ago. That would originally have run on a 40MHz ARM 7 machine with maybe 4Mb ram. As I recall I was sharing an office - actually a tiny fraction of an office cubicle - with John Maloney's wife Roxie at the time. How time flies.

/tim
{insert witticism here}

> On Jul 22, 2014, at 21:59, "David T. Lewis" <[hidden email]> wrote:
>
>
>> On Tue, Jul 22, 2014 at 08:55:49PM -0700, tim Rowledge wrote:
>>
>>> my assumption is that
>>> since Squeak began on big endian machines and moved to little endian
>>> later, and since the LargeIntegersPlugin has been around for a long time,
>>> and since images migrated across big and little endian platforms without
>>> any apparent problem, then we probably do not have any endianness problems
>>> with the existing plugin.
>>
>> There is, or at least was, code to byteswap byte objects at image load time. Not in a good spot to check right now. I think it went
>> Check endian is as was
>> Swap all words if needed.
>> Swap back contents of byte objects.
>>
>> Or something like that.
>>
>> /tim
>
> Judging by the author credited in the method comment, I suspect that you are probably right ;)
>
> Interpreter>>reverseBytesInImage
> "Byte-swap all words in memory after reading in the entire image file with bulk read.
> Contributed by Tim Rowledge."
>
> "First, byte-swap every word in the image. This fixes objects headers."
> objectMemory reverseBytesFrom: objectMemory startOfMemory to: objectMemory getEndOfMemory.
>
> "Second, return the bytes of bytes-type objects to their orginal order."
> self byteSwapByteObjects.
>
>

Nicolas Cellier-3

Re: Status of LargeIntegersPlugin

In reply to this post by David T. Lewis

2014-07-23 5:20 GMT+02:00 David T. Lewis <[hidden email]>:

Moving discussion back to vm-dev, as it may be of some general interest.

On Tue, Jul 22, 2014 at 09:16:23AM -0700, Eliot Miranda wrote:
> On Mon, Jul 21, 2014 at 5:35 PM, David T. Lewis <[hidden email]> wrote:
>
> > Hi Eliot,
> >
> > I want to check before I copy this update to the VMMaker repository - are
> > you in agreement with using Nicolas' LargeIntegerPlugin? It seems fine to
> > me, so I'll move it to VMM trunk if you agree.
> >
>
> Well, it is goodness. My concern is that it requires image changes to make
> image code still function, but haven't had time to check it out. The issue
> is that currently, large integers are in little endian order and code in
> the image accesses bytes in large integers assuming that the bytes there-in
> are in little endian order. For example see digitCompare: and digitAt:.
> Now digitAt: is implemented as simply primitive 60, the standard at:
> primitive. So on current VMs a big-endian large integer will /not/ be
> correctly accessed via digitAt:.
>
> Am I right?

I do not have a big endian machine to check this, but my assumption is that
since Squeak began on big endian machines and moved to little endian
later, and since the LargeIntegersPlugin has been around for a long time,
and since images migrated across big and little endian platforms without
any apparent problem, then we probably do not have any endianness problems
with the existing plugin.

I wrote a few tests (and added them to trunk) to validate the digit access
methods. These tests pass on my little endian PC with a Cog VM using the
old plugin, and they also pass with Nicolas' new LargeIntegersPlugin with
both the 32-bit and 64-bit image formats.

Nicolas' updated plugin seems to be fully compatible with the original
LargeIntegersPlugin, and it produces the expected results for all of the
image formats and VMs that I am able to test. I cannot confirm this for
a big endian platform, but I think that any errors on big endian box
would be exposed by the unit tests.

It looks good to me.

Nicolas, any concerns or comments?

Thanks,
Dave

There are two versions of my work

- one using 32 bits digits which is my original work commented on my blog http://smallissimo.blogspot.fr/2012/12/accelerating-largeinteger-in-squeak.html and following posts that you can find on MC repository http://smalltalkhub.com/mc/nice/NiceVMExperiments/main

- a backport of this work for 8 bit digits that I failed to commit due to zip bug, and that I sent as a change set

I didn't check, but presumably you integrated the last one for 8 bit digits.

So there should be no problem of endianness

Nicolas

David T. Lewis

Re: Status of LargeIntegersPlugin

On Wed, Jul 23, 2014 at 11:44:42AM +0200, Nicolas Cellier wrote:

>
> 2014-07-23 5:20 GMT+02:00 David T. Lewis <[hidden email]>:
> >
> > Moving discussion back to vm-dev, as it may be of some general interest.
> >
> > On Tue, Jul 22, 2014 at 09:16:23AM -0700, Eliot Miranda wrote:
> > > On Mon, Jul 21, 2014 at 5:35 PM, David T. Lewis <[hidden email]>
> > wrote:
> > >
> > > > Hi Eliot,
> > > >
> > > > I want to check before I copy this update to the VMMaker repository -
> > are
> > > > you in agreement with using Nicolas' LargeIntegerPlugin? It seems fine
> > to
> > > > me, so I'll move it to VMM trunk if you agree.
> > > >
> > >
> > > Well, it is goodness. My concern is that it requires image changes to
> > make
> > > image code still function, but haven't had time to check it out. The
> > issue
> > > is that currently, large integers are in little endian order and code in
> > > the image accesses bytes in large integers assuming that the bytes
> > there-in
> > > are in little endian order. For example see digitCompare: and digitAt:.
> > > Now digitAt: is implemented as simply primitive 60, the standard at:
> > > primitive. So on current VMs a big-endian large integer will /not/ be
> > > correctly accessed via digitAt:.
> > >
> > > Am I right?
> >
> > I do not have a big endian machine to check this, but my assumption is that
> > since Squeak began on big endian machines and moved to little endian
> > later, and since the LargeIntegersPlugin has been around for a long time,
> > and since images migrated across big and little endian platforms without
> > any apparent problem, then we probably do not have any endianness problems
> > with the existing plugin.
> >
> > I wrote a few tests (and added them to trunk) to validate the digit access
> > methods. These tests pass on my little endian PC with a Cog VM using the
> > old plugin, and they also pass with Nicolas' new LargeIntegersPlugin with
> > both the 32-bit and 64-bit image formats.
> >
> > Nicolas' updated plugin seems to be fully compatible with the original
> > LargeIntegersPlugin, and it produces the expected results for all of the
> > image formats and VMs that I am able to test. I cannot confirm this for
> > a big endian platform, but I think that any errors on big endian box
> > would be exposed by the unit tests.
> >
> > It looks good to me.
> >
> > Nicolas, any concerns or comments?
> >
> > Thanks,
> > Dave
> >
> >
> There are two versions of my work
> - one using 32 bits digits which is my original work commented on my blog
> http://smallissimo.blogspot.fr/2012/12/accelerating-largeinteger-in-squeak.html
> and following posts that you can find on MC repository
> http://smalltalkhub.com/mc/nice/NiceVMExperiments/main
> - a backport of this work for 8 bit digits that I failed to commit due to
> zip bug, and that I sent as a change set
> I didn't check, but presumably you integrated the last one for 8 bit digits.
> So there should be no problem of endianness

Nicolas,

Yes, I integrated the second one for 8 bit digits. I will go ahead and move it
into the VMMaker repository, and commit the ./src files for trunk SVN.

Thank you!

Dave