Smalltalk › Instantiations

VASmalltalk 64-bit bigger image and slower execution than 32-bit

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

10 messages Options

Klaus Breker-2

VASmalltalk 64-bit bigger image and slower execution than 32-bit

Hello,

in our project we are developing with both environments: 64 - and 32-bit VASmalltalk.
We are loading our full development configuration maps on a vanilla image (ibmstx.icx) without VisualAge stuff loaded.

Result:
32-bit image: 39756 kByte with Silvermark's Testmentor
64-bit image: 55352 kByte without Silvermark's Testmentor
Ok, 64-bit is needs more memory, so that the resulting image may be bigger than a 32-bit image.

Speed:
We measured the speed via 13000 SUnit tests without architecture dependent communication access on one machine with Windows 7.
Only Runtime Maps with SUnit Maps are loaded. No Silvermark's Testmentor or VisualAge Assist.

Results:
32-bit: 1992 seconds within 5 runs
64-bit: 2337 seconds within 5 runs

This is an average difference of 345 seconds. More than 5 minutes.
Is this normal? Has anyone made similar experiences?

Kind regards

Klaus

--
You received this message because you are subscribed to the Google Groups "VA Smalltalk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/va-smalltalk.
For more options, visit https://groups.google.com/d/optout.

Seth Berman

Re: VASmalltalk 64-bit bigger image and slower execution than 32-bit

Hello Klaus,

Yes, a 64-bit image is expected to be bigger because Objects and vm data structures are bigger.

The performance is not so easily answered. There are certain performance characteristics of the the 64-bit vm can be faster and others that can be slower. It all depends on what these tests are doing. Also, if it stresses memory allocation, are you setting up the heap appropriately with more space for the 64-bit to run in or are they both running in the same heap configuration...that would certainly have an impact. Do you have a lot of object swapping going on...by default the 64-bit vm will swap out a 32-bit compatible object dump...that would have an impact. Same bytecode set, but different bytecode mixture due to different offsets between the two...that has an impact.

We've got about 25 benchmarks that each give us a view of how the various parts of the vm machinery performs for both 32 and 64-bit. It is expected that the 64-bit vm should be a little slower on average (except our numeric prims tend to do better sometimes). In your case, it could be that the extra time is amortized across all 13000 tests...but it could be that you have hotspots in these tests that need to be identified or maybe its GC time and the heap needs some additional configuration for 64-bit.

Its hard to say what would account for that time without understanding what parts of the vm these tests are hammering and then profiling those parts.

Regards,

- Seth

On Thursday, January 25, 2018 at 10:37:40 AM UTC-5, Klaus Breker wrote:

Hello,

in our project we are developing with both environments: 64 - and 32-bit VASmalltalk Version 9.0 without the latest (first) fixpack.
We are loading our full development configuration maps on a vanilla image (ibmstx.icx) without VisualAge stuff loaded.

Result:
32-bit image: 39756 kByte with Silvermark's Testmentor
64-bit image: 55352 kByte without Silvermark's Testmentor
Ok, 64-bit is needs more memory, so that the resulting image may be bigger than a 32-bit image.

Speed:
We measured the speed via 13000 SUnit tests without architecture dependent communication access on one machine with Windows 7.
Only Runtime Maps with SUnit Maps are loaded. No Silvermark's Testmentor or VisualAge Assist.

Results:
32-bit: 1992 seconds within 5 runs
64-bit: 2337 seconds within 5 runs

This is an average difference of 345 seconds. More than 5 minutes.
Is this normal? Has anyone made similar experiences?

Kind regards

Klaus

Klaus Breker-2

Re: VASmalltalk 64-bit bigger image and slower execution than 32-bit

Hello Seth,

we are changing between German and English during testing our application strings. Every test method loads all our mpr and cat files. If this is object swapping, then we have the reason of the performance difference.

Regards

Klaus

Am Freitag, 26. Januar 2018 03:30:37 UTC+1 schrieb Seth Berman:

Hello Klaus,

Yes, a 64-bit image is expected to be bigger because Objects and vm data structures are bigger.

The performance is not so easily answered. There are certain performance characteristics of the the 64-bit vm can be faster and others that can be slower. It all depends on what these tests are doing. Also, if it stresses memory allocation, are you setting up the heap appropriately with more space for the 64-bit to run in or are they both running in the same heap configuration...that would certainly have an impact. Do you have a lot of object swapping going on...by default the 64-bit vm will swap out a 32-bit compatible object dump...that would have an impact. Same bytecode set, but different bytecode mixture due to different offsets between the two...that has an impact.

We've got about 25 benchmarks that each give us a view of how the various parts of the vm machinery performs for both 32 and 64-bit. It is expected that the 64-bit vm should be a little slower on average (except our numeric prims tend to do better sometimes). In your case, it could be that the extra time is amortized across all 13000 tests...but it could be that you have hotspots in these tests that need to be identified or maybe its GC time and the heap needs some additional configuration for 64-bit.

Its hard to say what would account for that time without understanding what parts of the vm these tests are hammering and then profiling those parts.

Regards,

- Seth

On Thursday, January 25, 2018 at 10:37:40 AM UTC-5, Klaus Breker wrote:
Hello,

in our project we are developing with both environments: 64 - and 32-bit VASmalltalk Version 9.0 without the latest (first) fixpack.
We are loading our full development configuration maps on a vanilla image (ibmstx.icx) without VisualAge stuff loaded.

Result:
32-bit image: 39756 kByte with Silvermark's Testmentor
64-bit image: 55352 kByte without Silvermark's Testmentor
Ok, 64-bit is needs more memory, so that the resulting image may be bigger than a 32-bit image.

Speed:
We measured the speed via 13000 SUnit tests without architecture dependent communication access on one machine with Windows 7.
Only Runtime Maps with SUnit Maps are loaded. No Silvermark's Testmentor or VisualAge Assist.

Results:
32-bit: 1992 seconds within 5 runs
64-bit: 2337 seconds within 5 runs

This is an average difference of 345 seconds. More than 5 minutes.
Is this normal? Has anyone made similar experiences?

Kind regards

Klaus

Seth Berman

Re: VASmalltalk 64-bit bigger image and slower execution than 32-bit

Hello Klaus,

Good find...that certainly seems the likely culprit those would be 32-bit serialized object files.

- Seth

--
You received this message because you are subscribed to the Google Groups "VA Smalltalk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/va-smalltalk.
For more options, visit https://groups.google.com/d/optout.

Klaus Breker-2

Re: VASmalltalk 64-bit bigger image and slower execution than 32-bit

Hello Seth,

I created with VASmalltalk (64-bit) our mpr and cat files. These files were accepted by our 32-bit-application. I assume, that 64-bit VASmalltalk creates 32-bit object files. Am I right?
If I am right, will the dumper in 64-bit VAASmalltalk in the future create 64-bit object files? And then I suppose, that this files will not be compatible with 32-bit Smalltalk.

Kind regards

Klaus

Am Donnerstag, 1. Februar 2018 03:29:08 UTC+1 schrieb Seth Berman:

Hello Klaus,
Good find...that certainly seems the likely culprit those would be 32-bit serialized object files.
- Seth

Seth Berman

Re: VASmalltalk 64-bit bigger image and slower execution than 32-bit

Hello Klaus,

I assume, that 64-bit VASmalltalk creates 32-bit object files. Am I right?

- Correct. The 64-bit vm and image understands what a 32-bit object and memory segment layout looks like and can load and save 32-bit object graphs

at the expense of some extra memory and processing time.

- The 64-bit image can recognize that it's loading a segment full of 32-bit objects and will auto convert these to a segment full of 64-bit objects on-the-fly.

- The 64-bit image dumps a translated 32-bit object graph, by default (more on that below).

- A 32-bit image (especially a <= 8.6.3 version) can NOT recognize a segment full of 64-bit objects and will attempt to load up what is effectively garbage in the 32-bit world. This will result in some form of general protection fault.

If I am right, will the dumper in 64-bit VAASmalltalk in the future create 64-bit object files?

- Yes, it can do it now. When creating an ObjectDumper, give it a EswDumpingStrategy with compatibiltyMode set to false.

- @see EswDumpingStrategy>>compatibilityMode:

And then I suppose, that this files will not be compatible with 32-bit Smalltalk.

- Correct. Not compatible with 9.0 32-bit. Not compatible, or even detectable, by <= 8.6.3 32-bit and will result in a hard crash if loading a 64-bit dump is attempted.

Kind Regards,

-- Seth

On Thursday, February 1, 2018 at 11:41:05 AM UTC-5, Klaus Breker wrote:

Hello Seth,

I created with VASmalltalk (64-bit) our mpr and cat files. These files were accepted by our 32-bit-application. I assume, that 64-bit VASmalltalk creates 32-bit object files. Am I right?
If I am right, will the dumper in 64-bit VAASmalltalk in the future create 64-bit object files? And then I suppose, that this files will not be compatible with 32-bit Smalltalk.

Kind regards

Klaus

Am Donnerstag, 1. Februar 2018 03:29:08 UTC+1 schrieb Seth Berman:
Hello Klaus,
Good find...that certainly seems the likely culprit those would be 32-bit serialized object files.
- Seth

Klaus Breker-2

Re: VASmalltalk 64-bit bigger image and slower execution than 32-bit

Hello Seth,

I see: EswDumpingStrategy->compatibilityMode: is set to true by default. In 32 - and 64-bit VASmalltalk.

Thanks a lot for your help and have a nice weekend.

Klaus

Am Donnerstag, 1. Februar 2018 22:14:29 UTC+1 schrieb Seth Berman:

Hello Klaus,

I assume, that 64-bit VASmalltalk creates 32-bit object files. Am I right?
- Correct. The 64-bit vm and image understands what a 32-bit object and memory segment layout looks like and can load and save 32-bit object graphs
at the expense of some extra memory and processing time.
- The 64-bit image can recognize that it's loading a segment full of 32-bit objects and will auto convert these to a segment full of 64-bit objects on-the-fly.
- The 64-bit image dumps a translated 32-bit object graph, by default (more on that below).
- A 32-bit image (especially a <= 8.6.3 version) can NOT recognize a segment full of 64-bit objects and will attempt to load up what is effectively garbage in the 32-bit world. This will result in some form of general protection fault.

If I am right, will the dumper in 64-bit VAASmalltalk in the future create 64-bit object files?
- Yes, it can do it now. When creating an ObjectDumper, give it a EswDumpingStrategy with compatibiltyMode set to false.
- @see EswDumpingStrategy>>compatibilityMode:

And then I suppose, that this files will not be compatible with 32-bit Smalltalk.
- Correct. Not compatible with 9.0 32-bit. Not compatible, or even detectable, by <= 8.6.3 32-bit and will result in a hard crash if loading a 64-bit dump is attempted.

Kind Regards,

-- Seth

On Thursday, February 1, 2018 at 11:41:05 AM UTC-5, Klaus Breker wrote:
Hello Seth,

I created with VASmalltalk (64-bit) our mpr and cat files. These files were accepted by our 32-bit-application. I assume, that 64-bit VASmalltalk creates 32-bit object files. Am I right?
If I am right, will the dumper in 64-bit VAASmalltalk in the future create 64-bit object files? And then I suppose, that this files will not be compatible with 32-bit Smalltalk.

Kind regards

Klaus

Am Donnerstag, 1. Februar 2018 03:29:08 UTC+1 schrieb Seth Berman:
Hello Klaus,
Good find...that certainly seems the likely culprit those would be 32-bit serialized object files.
- Seth

Seth Berman

Re: VASmalltalk 64-bit bigger image and slower execution than 32-bit

My pleasure Klaus...have a good weekend, as well.

-- Seth

On Friday, February 2, 2018 at 2:30:32 AM UTC-5, Klaus Breker wrote:

Hello Seth,

I see: EswDumpingStrategy->compatibilityMode: is set to true by default. In 32 - and 64-bit VASmalltalk.

Thanks a lot for your help and have a nice weekend.

Klaus

Am Donnerstag, 1. Februar 2018 22:14:29 UTC+1 schrieb Seth Berman:
Hello Klaus,

I assume, that 64-bit VASmalltalk creates 32-bit object files. Am I right?
- Correct. The 64-bit vm and image understands what a 32-bit object and memory segment layout looks like and can load and save 32-bit object graphs
at the expense of some extra memory and processing time.
- The 64-bit image can recognize that it's loading a segment full of 32-bit objects and will auto convert these to a segment full of 64-bit objects on-the-fly.
- The 64-bit image dumps a translated 32-bit object graph, by default (more on that below).
- A 32-bit image (especially a <= 8.6.3 version) can NOT recognize a segment full of 64-bit objects and will attempt to load up what is effectively garbage in the 32-bit world. This will result in some form of general protection fault.

If I am right, will the dumper in 64-bit VAASmalltalk in the future create 64-bit object files?
- Yes, it can do it now. When creating an ObjectDumper, give it a EswDumpingStrategy with compatibiltyMode set to false.
- @see EswDumpingStrategy>>compatibilityMode:

And then I suppose, that this files will not be compatible with 32-bit Smalltalk.
- Correct. Not compatible with 9.0 32-bit. Not compatible, or even detectable, by <= 8.6.3 32-bit and will result in a hard crash if loading a 64-bit dump is attempted.

Kind Regards,

-- Seth

On Thursday, February 1, 2018 at 11:41:05 AM UTC-5, Klaus Breker wrote:
Hello Seth,

I created with VASmalltalk (64-bit) our mpr and cat files. These files were accepted by our 32-bit-application. I assume, that 64-bit VASmalltalk creates 32-bit object files. Am I right?
If I am right, will the dumper in 64-bit VAASmalltalk in the future create 64-bit object files? And then I suppose, that this files will not be compatible with 32-bit Smalltalk.

Kind regards

Klaus

Am Donnerstag, 1. Februar 2018 03:29:08 UTC+1 schrieb Seth Berman:
Hello Klaus,
Good find...that certainly seems the likely culprit those would be 32-bit serialized object files.
- Seth

Klaus Breker-2

Re: VASmalltalk 64-bit bigger image and slower execution than 32-bit

Hello,

we made an extension to #ObjectDumper to set #EswDumpingStrategy -> #compatibilityMode: to false for creating 64-bit mpr-files.
With this files our SUnit tests ran as fast as the 32-bit SUnit tests in the 32-bit development environment. Nice.
But then I forgot to reset the compatibility mode, made changes in a part (sublass of #AbtAppBldrView), saved and released this part.
When I tried to open this part in the composition editor in our 32-bit environment, a debugger raised with a primitive error.

Error message: 'Primitive failed in: EsMemorySegment class>>#createObjectsFromBytes:originalPlatformFlags:classesFixup:copyInstances:variableByteFixup:variablePointerFixup:encodedClasses: due to Unimplemented primitive

I think, it is no good idea to create mpr - or cat-files in 64-bit without compatibility to 32-bit, when I have to develop in a 32-bit environment too.

Kind regards

Seth Berman

Re: VASmalltalk 64-bit bigger image and slower execution than 32-bit

Hello Klaus,

Yes, I just looked and a 9.0 32-bit will kindly check if the 64-bit flag is present on a serialized object stream and raise this error.
If you tried to do this in <= 8.6.3...the result would be a GPF hard crash with no indication of why (I learned this lesson well:)

I would only recommend using this setting in localized instances only. For example, if you have an object you want to dump out and will be constructing your own ObjectDumper.

Anything compatibility setting changes to the default policy will result in changes out of your control.

For example, Sts development tools saves it's settings in the ENVY manager as serialized objects using Object Dumper and it's default policy. There are lots of other places where this may happen as well.

As you have found out, you don't want to be in a position where 64-bit serialized objects are in the manager...this will guarantee that 32-bit images connected to the same manager will not be able to load it and inexplicably crash shortly after starting a 32-bit image (i.e. loading up sts power tool settings on startup).

Kind Regards,

-- Seth

On Wednesday, February 14, 2018 at 11:25:01 AM UTC-5, Klaus Breker wrote:

Hello,

we made an extension to #ObjectDumper to set #EswDumpingStrategy -> #compatibilityMode: to false for creating 64-bit mpr-files.
With this files our SUnit tests ran as fast as the 32-bit SUnit tests in the 32-bit development environment. Nice.
But then I forgot to reset the compatibility mode, made changes in a part (sublass of #AbtAppBldrView), saved and released this part.
When I tried to open this part in the composition editor in our 32-bit environment, a debugger raised with a primitive error.

Error message: 'Primitive failed in: EsMemorySegment class>>#createObjectsFromBytes:originalPlatformFlags:classesFixup:copyInstances:variableByteFixup:variablePointerFixup:encodedClasses: due to Unimplemented primitive

I think, it is no good idea to create mpr - or cat-files in 64-bit without compatibility to 32-bit, when I have to develop in a 32-bit environment too.

Kind regards