Smalltalk › Squeak › Squeak VM

[OpenSmalltalk/opensmalltalk-vm] Desired size of Eden parameter is not respected (#476)

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

4 messages Options

David T Lewis

[OpenSmalltalk/opensmalltalk-vm] Desired size of Eden parameter is not respected (#476)

I want to customize the GC parameters for images that will probably grow to be very large.

One of the most important performance-wise is the size of the Eden. IIUR, the size cannot be changed at runtime so there are two VM parameters. The parameter 44 gives the current size of the Eden while parameter 45 should be used to set the desired size of the Eden.

What I did was that I executed this:

$VM Pharo.image --no-default-preferences eval --save "Smalltalk vm parameterAt: 45 put: 67108864. Smalltalk vm parameterAt: 25 put: 33554432. Smalltalk vm parameterAt: 24 put: 67108864. Smalltalk vm parameterAt: 55 put: 0.7. 'GC tunned'"
$VM Pharo.image

In the image that opens I then get this:

Smalltalk vm parameterAt: 44. "49546912"
Smalltalk vm parameterAt: 45. "67108864"

There is quite a large margin between the two.

I got the VM via: https://files.pharo.org/get-files/pharo64-mac-stable.zip

Image
-----
/Users/cyrilferlicot/Downloads/Pharo/Pharo.image
Pharo8.0.0
Build information: Pharo-8.0.0+build.1128.sha.9f6475d88dda7d83acdeeda794df35d304cf620d (64 Bit)
Unnamed

Virtual Machine
---------------
/Users/cyrilferlicot/Downloads/Pharo/pharo-mac-vm/Pharo.app/Contents/MacOS/Pharo
CoInterpreter VMMaker.oscog-eem.2509 uuid: 91e81f64-95de-4914-a960-8f842be3a194 Feb  7 2019
StackToRegisterMappingCogit VMMaker.oscog-eem.2509 uuid: 91e81f64-95de-4914-a960-8f842be3a194 Feb  7 2019
VM: 201902062351 https://github.com/OpenSmalltalk/opensmalltalk-vm.git Date: Wed Feb 6 15:51:18 2019 CommitHash: a838346 Plugins: 201902062351 https://github.com/OpenSmalltalk/opensmalltalk-vm.git

Mac OS X built on Feb  7 2019 00:01:47 UTC Compiler: 4.2.1 Compatible Apple LLVM 7.3.0 (clang-703.0.31)
VMMaker versionString VM: 201902062351 https://github.com/OpenSmalltalk/opensmalltalk-vm.git Date: Wed Feb 6 15:51:18 2019 CommitHash: a838346 Plugins: 201902062351 https://github.com/OpenSmalltalk/opensmalltalk-vm.git
CoInterpreter VMMaker.oscog-eem.2509 uuid: 91e81f64-95de-4914-a960-8f842be3a194 Feb  7 2019
StackToRegisterMappingCogit VMMaker.oscog-eem.2509 uuid: 91e81f64-95de-4914-a960-8f842be3a194 Feb  7 2019

Virtual Machine Commandline Options
-----------------------------------
none

Virtual Machine Parameters
--------------------------
#1	1297188640	end (v3)/size(Spur) of old-space (0-based, read-only)
#2	8941328	end (v3)/size(Spur) of young/new-space (read-only)
#3	1365909504	end (v3)/size(Spur) of heap (read-only)
#4	nil	nil (was allocationCount (read-only))
#5	nil	nil (was allocations between GCs (read-write)
#6	0	survivor count tenuring threshold (read-write)
#7	4	full GCs since startup (read-only)
#8	9937	total milliseconds in full GCs since startup (read-only)
#9	71	incremental GCs (SqueakV3) or scavenges (Spur) since startup (read-only)
#10	77	total milliseconds in incremental GCs (SqueakV3) or scavenges (Spur) since startup (read-only)
#11	0	tenures of surving objects since startup (read-only)
#12	0	12-20 were specific to ikp's JITTER VM, now 12-19 are open for use
#13	0	12-20 were specific to ikp's JITTER VM, now 12-19 are open for use
#14	0	12-20 were specific to ikp's JITTER VM, now 12-19 are open for use
#15	0	12-20 were specific to ikp's JITTER VM, now 12-19 are open for use
#16	1053071685	12-20 were specific to ikp's JITTER VM, now 12-19 are open for use
#17	0	12-20 were specific to ikp's JITTER VM, now 12-19 are open for use
#18	3361	12-20 were specific to ikp's JITTER VM, now 12-19 are open for use
#19	47160742	12-20 were specific to ikp's JITTER VM, now 12-19 are open for use
#20	3759644577117121	utc microseconds at VM start-up (actually at time initialization, which precedes image load).
#21	348	root table size (read-only)
#22	0	root table overflows since startup (read-only)
#23	0	bytes of extra memory to reserve for VM buffers, plugins, etc (stored in image file header).
#24	33554432	memory threshold above which shrinking object memory (rw)
#25	16777216	memory headroom when growing object memory (rw)
#26	2	interruptChecksEveryNms - force an ioProcessEvents every N milliseconds	(rw) 27	number of times mark loop iterated for current IGC/FGC (read-only)	includes ALL marking
#28	0	number of times sweep loop iterated for current IGC/FGC (read-only)
#29	0	number of times make forward loop iterated for current IGC/FGC	(read-only) 30	number of times compact move loop iterated for current	IGC/FGC (read-only)
#31	2	number of grow memory requests (read-only)
#32	0	number of shrink memory requests (read-only)
#33	343	number of root table entries used for current IGC/FGC (read-only)
#34	3420031008	number of allocations done before current IGC/FGC (read-only)
#35	39534	number of survivor objects after current IGC/FGC (read-only)
#36	475582454	millisecond clock when current IGC/FGC completed (read-only)
#37	0	number of marked objects for Roots of the world, not including Root	Table entries for current IGC/FGC (read-only)
#38	0	milliseconds taken by current IGC (read-only)
#39	0	Number of finalization signals for Weak Objects pending when current	IGC/FGC completed (read-only)
#40	8	BytesPerOop for this image
#41	68021	imageFormatVersion for the VM
#42	50	number of stack pages in use
#43	0	desired number of stack pages (stored in image file header, max 65535)
#44	49546912	size of eden, in bytes
#45	67108864	desired size of eden, in bytes (stored in image file header)
#46	1433600	machine code zone size, in bytes (Cog only; otherwise nil)
#47	0	desired machine code zone size (stored in image file header; Cog only;	otherwise nil)
#48	0	various header flags. See getCogVMFlags.
#49	256	max size the image promises to grow the external semaphore table to (0	sets to default, which is 256 as of writing)
#50	nil	nil; reserved for VM parameters that persist in the image (such as	eden above)
#51	nil	nil; reserved for VM parameters that persist in the image (such as	eden above)
#52	524288	root table capacity
#53	3	number of segments (Spur only; otherwise nil)
#54	21608328	total size of free old space (Spur only, otherwise nil)
#55	0.33333298563957214	ratio of growth and image size at or above which a GC will be performed	post scavenge
#56	266089	number of process switches since startup (read-only)
#57	50406	number of ioProcessEvents calls since startup (read-only)
#58	518425	number of ForceInterruptCheck calls since startup (read-only)
#59	86052	number of check event calls since startup (read-only)
#60	504221	number of stack page overflows since startup (read-only)
#61	0	number of stack page divorces since startup (read-only)	62	compiled code compactions since startup (read-only; Cog only; otherwise nil)
#63	26	total milliseconds in compiled code compactions since startup	(read-only; Cog only; otherwise nil)
#64	3035	the number of methods that currently have jitted machine-code
#65	3	whether the VM supports a certain feature, MULTIPLE_BYTECODE_SETS is bit 0, IMMTABILITY is bit 1
#66	4096	the byte size of a stack page
#67	0	the max allowed size of old space (Spur only; nil otherwise; 0 implies	no limit except that of the underlying platform)
#68	15.455696202531646	the average number of live stack pages when scanned by GC (at	scavenge/gc/become et al)
#69	21	the maximum number of live stack pages when scanned by GC (at	scavenge/gc/become et al)
#70	1	the vmProxyMajorVersion (the interpreterProxy VM_MAJOR_VERSION)
#71	15	the vmProxyMinorVersion (the interpreterProxy VM_MINOR_VERSION)

Operating System/Hardware
-------------------------
Mac OS 1014.6 x64

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

David T Lewis

Re: [OpenSmalltalk/opensmalltalk-vm] Desired size of Eden parameter is not respected (#476)

Thank you for the info!

I'll close the issue here and open one in the Pharo issue tracker to update the descriptions of the parameters.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or unsubscribe.

David T Lewis

Re: [OpenSmalltalk/opensmalltalk-vm] Desired size of Eden parameter is not respected (#476)

In reply to this post by David T Lewis

Closed #476.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or unsubscribe.

johnmci

Re: [OpenSmalltalk/opensmalltalk-vm] Desired size of Eden parameter is not respected (#476)

In reply to this post by David T Lewis

Having control over eden and survivor spaces independently would be nice. I have in the real world in another smalltalk that uses Ungar's memory model solved a nasty bit of earlier tenuring by tweaking the survivor space size 'just right'.

....

John M. McIntosh. Corporate Smalltalk Consulting Ltd https://www.linkedin.com/in/smalltalk

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐

On Thursday, February 20, 2020 3:07 AM, Eliot Miranda <[hidden email]> wrote:

Hi Cyril,

On Feb 20, 2020, at 2:03 AM, CyrilFerlicot <[hidden email]> wrote:

I want to customize the GC parameters for images that will probably grow to be very large.
One of the most important performance-wise is the size of the Eden. IIUR, the size cannot be changed at runtime so there are two VM parameters. The parameter 44 gives the current size of the Eden while parameter 45 should be used to set the desired size of the Eden.
What I did was that I executed this:
$VM Pharo.image --no-default-preferences eval --save "Smalltalk vm parameterAt: 45 put: 67108864. Smalltalk vm parameterAt: 25 put: 33554432. Smalltalk vm parameterAt: 24 put: 67108864. Smalltalk vm parameterAt: 55 put: 0.7. 'GC tunned'" $VM Pharo.image
In the image that opens I then get this:
Smalltalk vm parameterAt: 44. "49546912" Smalltalk vm parameterAt: 45. "67108864"
There is quite a large margin between the two.

Alas this discrepancy is historical and semantic. In the old V3 memory manager there was only old space and eden, and one could set the size of eden dynamically because it was simply “the region at the end of old space”, marked by a base/limit paid of pointers (IIRC; I’m on my phone so I might have details wrong).

In Spur eden is actually one of three regions of new space, the other two being the two survivor spaces, past survivor space and future survivor space (past space and future space for short). Past and future space are the same size and swap after each scavenge, which empties eden and past space, copying survivors into future space, tenuring overflow to old space and then making future space the new past space. The ratio of each survivor space to eden is 1 to 5, i.e. eden is 5/7 of new space and the survivor spaces are 2/7 of new space (currently this ratio cannot be changed).

What we really want to do is set the size of new space but the parameter historically referred to v3’s eden. So Spur interprets parameter 45 as setting the desired size of (all of) new space but accurately reports the size of eden for parameter 44. (49546912 / 67108864) ≈ (5/7).

So the vm parameter methods should state that vm parameter 45 in v3 sets the desired size of eden and in Spur sets the desired size of new space, while in both, parameter 44 reports the size of eden. And yes, this is a bit of a mess. IIRC there is another parameter that reports the size of new space, but it may report the occupancy of new space.

HTH

_,,,^..^,,,_ (phone)

I got the VM via: https://files.pharo.org/get-files/pharo64-mac-stable.zip

Image
-----
/Users/cyrilferlicot/Downloads/Pharo/Pharo.image
Pharo8.0.0
Build information: Pharo-8.0.0+build.1128.sha.9f6475d88dda7d83acdeeda794df35d304cf620d (64 Bit)
Unnamed

Virtual Machine
---------------
/Users/cyrilferlicot/Downloads/Pharo/pharo-mac-vm/Pharo.app/Contents/MacOS/Pharo
CoInterpreter VMMaker.oscog-eem.2509 uuid: 91e81f64-95de-4914-a960-8f842be3a194 Feb  7 2019
StackToRegisterMappingCogit VMMaker.oscog-eem.2509 uuid: 91e81f64-95de-4914-a960-8f842be3a194 Feb  7 2019
VM: 201902062351 https://github.com/OpenSmalltalk/opensmalltalk-vm.git Date: Wed Feb 6 15:51:18 2019 CommitHash: a838346 Plugins: 201902062351 https://github.com/OpenSmalltalk/opensmalltalk-vm.git

Mac OS X built on Feb  7 2019 00:01:47 UTC Compiler: 4.2.1 Compatible Apple LLVM 7.3.0 (clang-703.0.31)
VMMaker versionString VM: 201902062351 https://github.com/OpenSmalltalk/opensmalltalk-vm.git Date: Wed Feb 6 15:51:18 2019 CommitHash: a838346 Plugins: 201902062351 https://github.com/OpenSmalltalk/opensmalltalk-vm.git
CoInterpreter VMMaker.oscog-eem.2509 uuid: 91e81f64-95de-4914-a960-8f842be3a194 Feb  7 2019
StackToRegisterMappingCogit VMMaker.oscog-eem.2509 uuid: 91e81f64-95de-4914-a960-8f842be3a194 Feb  7 2019

Virtual Machine Commandline Options
-----------------------------------
none

Virtual Machine Parameters
--------------------------
#1	1297188640	end (v3)/size(Spur) of old-space (0-based, read-only)
#2	8941328	end (v3)/size(Spur) of young/new-space (read-only)
#3	1365909504	end (v3)/size(Spur) of heap (read-only)
#4	nil	nil (was allocationCount (read-only))
#5	nil	nil (was allocations between GCs (read-write)
#6	0	survivor count tenuring threshold (read-write)
#7	4	full GCs since startup (read-only)
#8	9937	total milliseconds in full GCs since startup (read-only)
#9	71	incremental GCs (SqueakV3) or scavenges (Spur) since startup (read-only)
#10	77	total milliseconds in incremental GCs (SqueakV3) or scavenges (Spur) since startup (read-only)
#11	0	tenures of surving objects since startup (read-only)
#12	0	12-20 were specific to ikp's JITTER VM, now 12-19 are open for use
#13	0	12-20 were specific to ikp's JITTER VM, now 12-19 are open for use
#14	0	12-20 were specific to ikp's JITTER VM, now 12-19 are open for use
#15	0	12-20 were specific to ikp's JITTER VM, now 12-19 are open for use
#16	1053071685	12-20 were specific to ikp's JITTER VM, now 12-19 are open for use
#17	0	12-20 were specific to ikp's JITTER VM, now 12-19 are open for use
#18	3361	12-20 were specific to ikp's JITTER VM, now 12-19 are open for use
#19	47160742	12-20 were specific to ikp's JITTER VM, now 12-19 are open for use
#20	3759644577117121	utc microseconds at VM start-up (actually at time initialization, which precedes image load).
#21	348	root table size (read-only)
#22	0	root table overflows since startup (read-only)
#23	0	bytes of extra memory to reserve for VM buffers, plugins, etc (stored in image file header).
#24	33554432	memory threshold above which shrinking object memory (rw)
#25	16777216	memory headroom when growing object memory (rw)
#26	2	interruptChecksEveryNms - force an ioProcessEvents every N milliseconds	(rw) 27	number of times mark loop iterated for current IGC/FGC (read-only)	includes ALL marking
#28	0	number of times sweep loop iterated for current IGC/FGC (read-only)
#29	0	number of times make forward loop iterated for current IGC/FGC	(read-only) 30	number of times compact move loop iterated for current	IGC/FGC (read-only)
#31	2	number of grow memory requests (read-only)
#32	0	number of shrink memory requests (read-only)
#33	343	number of root table entries used for current IGC/FGC (read-only)
#34	3420031008	number of allocations done before current IGC/FGC (read-only)
#35	39534	number of survivor objects after current IGC/FGC (read-only)
#36	475582454	millisecond clock when current IGC/FGC completed (read-only)
#37	0	number of marked objects for Roots of the world, not including Root	Table entries for current IGC/FGC (read-only)
#38	0	milliseconds taken by current IGC (read-only)
#39	0	Number of finalization signals for Weak Objects pending when current	IGC/FGC completed (read-only)
#40	8	BytesPerOop for this image
#41	68021	imageFormatVersion for the VM
#42	50	number of stack pages in use
#43	0	desired number of stack pages (stored in image file header, max 65535)
#44	49546912	size of eden, in bytes
#45	67108864	desired size of eden, in bytes (stored in image file header)
#46	1433600	machine code zone size, in bytes (Cog only; otherwise nil)
#47	0	desired machine code zone size (stored in image file header; Cog only;	otherwise nil)
#48	0	various header flags. See getCogVMFlags.
#49	256	max size the image promises to grow the external semaphore table to (0	sets to default, which is 256 as of writing)
#50	nil	nil; reserved for VM parameters that persist in the image (such as	eden above)
#51	nil	nil; reserved for VM parameters that persist in the image (such as	eden above)
#52	524288	root table capacity
#53	3	number of segments (Spur only; otherwise nil)
#54	21608328	total size of free old space (Spur only, otherwise nil)
#55	0.33333298563957214	ratio of growth and image size at or above which a GC will be performed	post scavenge
#56	266089	number of process switches since startup (read-only)
#57	50406	number of ioProcessEvents calls since startup (read-only)
#58	518425	number of ForceInterruptCheck calls since startup (read-only)
#59	86052	number of check event calls since startup (read-only)
#60	504221	number of stack page overflows since startup (read-only)
#61	0	number of stack page divorces since startup (read-only)	62	compiled code compactions since startup (read-only; Cog only; otherwise nil)
#63	26	total milliseconds in compiled code compactions since startup	(read-only; Cog only; otherwise nil)
#64	3035	the number of methods that currently have jitted machine-code
#65	3	whether the VM supports a certain feature, MULTIPLE_BYTECODE_SETS is bit 0, IMMTABILITY is bit 1
#66	4096	the byte size of a stack page
#67	0	the max allowed size of old space (Spur only; nil otherwise; 0 implies	no limit except that of the underlying platform)
#68	15.455696202531646	the average number of live stack pages when scanned by GC (at	scavenge/gc/become et al)
#69	21	the maximum number of live stack pages when scanned by GC (at	scavenge/gc/become et al)
#70	1	the vmProxyMajorVersion (the interpreterProxy VM_MAJOR_VERSION)
#71	15	the vmProxyMinorVersion (the interpreterProxy VM_MINOR_VERSION)

Operating System/Hardware
-------------------------
Mac OS 1014.6 x64

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or unsubscribe.