VM Maker: VMMaker.oscog-eem.2820.mcz

 
Eliot Miranda uploaded a new version of VMMaker to project VM Maker:
http://source.squeak.org/VMMaker/VMMaker.oscog-eem.2820.mcz

==================== Summary ====================

Name: VMMaker.oscog-eem.2820
Author: eem
Time: 25 September 2020, 10:07:31.980796 am
UUID: 6c06ad6c-bfd5-4d36-82af-c12df1ae5ad5
Ancestors: VMMaker.oscog-eem.2819

Spur:
Allow sufficientSpaceAfterGC: (which is invoked from checkForEventsMayContextSwitch:) to attempt to shrink if there is lots of free space.  sufficientSpaceAfterGC: invokes fullGC only if the heap has grown by the growth rarion since the last fullGC, and (before this change) only fullGC would attempt to shrink.  So memory would never shrink if some large amout of space became available, sicne the heap would not grow again, and fullGC would never be called.  hence the free space would remain in the system.  So this is needed to shrink when possible.

Have growOldSpaceByAtLeast: set the needGC flag if growth is either disallowed or fails.  This will cause the VM to invoke sufficientSpaceAfterGC: in a context of continuous growth that comes to a halt before, say, the growth ratio is reached.

Simulator:
Allow avoiding halting on deleting files.
Limit the size of the simulated heap by default (512Mb 64-bits, 256Mb 32-bits).  Why?  here's why:

Virtual Machine Statistics
--------------------------
uptime 10h 10m 23s (runtime 5h 15m 3s, idletime 4h 55m 19s)
memory 34,770,395,136 bytes
        old 34,761,443,104 bytes (100%)
        young 7,190,528 bytes (0%)
        used 17,596,793,984 bytes (50.6%)
        free 17,165,908,320 bytes (49.4%)
GCs 997,239 (36.7 ms between GCs 19 ms runtime between GCs)
        full 4,935 totalling 769,901 ms (4.07% runtime), avg 156 ms
                                marking 284,119 ms (36.9%) avg 57.6 ms,
                                compacting 485,782 ms (63.1%) avg 98.4 ms
        scavenges 992,304 totalling 151,685 ms (0.8% runtime), avg 0.2 ms
        tenures 2,834,722 (avg 2 tenures per scavenge)
Code compactions
        2,304 totalling 1,910 ms (0.01% runtime), avg 0.8 ms

=============== Diff against VMMaker.oscog-eem.2819 ===============

Item was changed:
  ----- Method: FilePluginSimulator>>sqFileDeleteName:Size: (in category 'simulation') -----
  sqFileDeleteName: nameIndex Size: nameSize
  | path |
  path := (interpreterProxy asString: nameIndex size: nameSize) asByteArray utf8Decoded.
  (StandardFileStream isAFileNamed: path) ifFalse:
  [^interpreterProxy primitiveFail].
+ (InitializationOptions at: #haltOnFileDelete ifAbsent: [true])
+ ifTrue: [self halt: 'Deleting ', (path contractTo: 64)]
+ ifFalse: [interpreterProxy transcript cr; show: 'Deleting ', (path contractTo: 64); cr].
- self halt: 'Deleting ', (path contractTo: 64).
  [FileDirectory deleteFilePath: path]
  on: Error
  do: [:ex| interpreterProxy primitiveFail]!

Item was changed:
  ----- Method: SpurMemoryManager>>growOldSpaceByAtLeast: (in category 'growing/shrinking memory') -----
  growOldSpaceByAtLeast: minAmmount
  "Attempt to grow memory by at least minAmmount.
  Answer the size of the new segment, or nil if the attempt failed."
  | ammount headroom total start interval |
  <var: #segInfo type: #'SpurSegmentInfo *'>
  "statGrowMemory counts attempts, not successes."
  statGrowMemory := statGrowMemory + 1."we need to include overhead for a new object header plus the segment bridge."
  ammount := minAmmount + (self baseHeaderSize * 2 + self bridgeSize).
  "round up to the nearest power of two."
  ammount := 1 << (ammount - 1) highBit.
  "and grow by at least growHeadroom."
  ammount := ammount max: growHeadroom.
 
  "Now apply the maxOldSpaceSize limit, if one is in effect."
  maxOldSpaceSize > 0 ifTrue:
  [total := segmentManager totalBytesInSegments.
  total >= maxOldSpaceSize ifTrue:
+ [needGCFlag := true.
+ ^nil].
- [^nil].
  headroom := maxOldSpaceSize - total.
  headroom < ammount ifTrue:
  [headroom < (minAmmount + (self baseHeaderSize * 2 + self bridgeSize)) ifTrue:
+ [needGCFlag := true.
+ ^nil].
- [^nil].
  ammount := headroom]].
 
  start := coInterpreter ioUTCMicrosecondsNow.
+ ^(segmentManager addSegmentOfSize: ammount)
+ ifNil: [needGCFlag := true. nil]
+ ifNotNil:
+ [:segInfo|
+ self assimilateNewSegment: segInfo.
+ "and add the new free chunk to the free list; done here
+  instead of in assimilateNewSegment: for the assert"
+ self addFreeChunkWithBytes: segInfo segSize - self bridgeSize at: segInfo segStart.
+ self assert: (self addressAfter: (self objectStartingAt: segInfo segStart))
+ = (segInfo segLimit - self bridgeSize).
+ self checkFreeSpace: GCModeFreeSpace.
+ segmentManager checkSegments.
+ interval := coInterpreter ioUTCMicrosecondsNow - start.
+ interval > statMaxAllocSegmentTime ifTrue: [statMaxAllocSegmentTime := interval].
+ segInfo segSize]!
- ^(segmentManager addSegmentOfSize: ammount) ifNotNil:
- [:segInfo|
- self assimilateNewSegment: segInfo.
- "and add the new free chunk to the free list; done here
-  instead of in assimilateNewSegment: for the assert"
- self addFreeChunkWithBytes: segInfo segSize - self bridgeSize at: segInfo segStart.
- self assert: (self addressAfter: (self objectStartingAt: segInfo segStart))
- = (segInfo segLimit - self bridgeSize).
- self checkFreeSpace: GCModeFreeSpace.
- segmentManager checkSegments.
- interval := coInterpreter ioUTCMicrosecondsNow - start.
- interval > statMaxAllocSegmentTime ifTrue: [statMaxAllocSegmentTime := interval].
- segInfo segSize]!

Item was changed:
  ----- Method: SpurMemoryManager>>initialize (in category 'initialization') -----
  initialize
  "We can put all initializations that set something to 0 or to false here.
  In C all global variables are initialized to 0, and 0 is false."
+ | moreThanEnough |
  remapBuffer := Array new: RemapBufferSize.
  remapBufferCount := extraRootCount := 0. "see below"
  freeListsMask := totalFreeOldSpace := lowSpaceThreshold := 0.
  checkForLeaks := 0.
  needGCFlag := signalLowSpace := marking := false.
  becomeEffectsFlags := gcPhaseInProgress := validatedIntegerClassFlags := 0.
  statScavenges := statIncrGCs := statFullGCs := 0.
  statMaxAllocSegmentTime := 0.
  statMarkUsecs := statSweepUsecs := statScavengeGCUsecs := statIncrGCUsecs := statFullGCUsecs := statCompactionUsecs := statGCEndUsecs := gcSweepEndUsecs := 0.
  statSGCDeltaUsecs := statIGCDeltaUsecs := statFGCDeltaUsecs := 0.
  statGrowMemory := statShrinkMemory := statRootTableCount := statAllocatedBytes := 0.
  statRootTableOverflows := statMarkCount := statCompactPassCount := statCoalesces := 0.
 
  "We can initialize things that are allocated but are lazily initialized."
  unscannedEphemerons := SpurContiguousObjStack new.
 
  "we can initialize things that are virtual in C."
  scavenger := SpurGenerationScavenger simulatorClass new manager: self; yourself.
  segmentManager := SpurSegmentManager simulatorClass new manager: self; yourself.
  compactor := self class compactorClass simulatorClass new manager: self; yourself.
 
  "We can also initialize here anything that is only for simulation."
  heapMap := CogCheck32BitHeapMap new.
 
  "N.B. We *don't* initialize extraRoots because we don't simulate it."
+
+ "This is needed on 64-bits. We don't want a simulation creating a huge heap by default.
+ By default use 512Mb on 64-bits, 256Mb on 32-bits."
+ moreThanEnough := 1024 * 1024 * 1024 / (16 / self wordSize). "One million dollars, ha ha ha ha ha,... ha, ha ha ha ha, ..."
  maxOldSpaceSize := self class initializationOptions
+ ifNotNil: [:initOpts| initOpts at: #maxOldSpaceSize ifAbsent: [moreThanEnough]]
+ ifNil: [moreThanEnough]!
- ifNotNil: [:initOpts| initOpts at: #maxOldSpaceSize ifAbsent: [0]]
- ifNil: [0]!

Item was changed:
  ----- Method: SpurMemoryManager>>sufficientSpaceAfterGC: (in category 'gc - scavenging') -----
  sufficientSpaceAfterGC: numBytes
  "This is ObjectMemory's funky entry-point into its incremental GC,
  which is a stop-the-world a young generation reclaimer.  In Spur
  we run the scavenger.  Answer if space is not low."
 
  | heapSizePostGC |
  self assert: numBytes = 0.
  self scavengingGCTenuringIf: TenureByAge.
  heapSizePostGC := segmentManager totalOldSpaceCapacity - totalFreeOldSpace.
+ (heapSizePostGC - heapSizeAtPreviousGC) asFloat / heapSizeAtPreviousGC >= heapGrowthToSizeGCRatio
+ ifTrue: [self fullGC] "fullGC will attempt to shrink"
+ ifFalse: "Also attempt to shrink if there is plenty of free space and no need to GC"
+ [totalFreeOldSpace > (shrinkThreshold * 2) ifTrue:
+ [self attemptToShrink.
+ ^true]].
- (heapSizePostGC - heapSizeAtPreviousGC) asFloat / heapSizeAtPreviousGC >= heapGrowthToSizeGCRatio ifTrue:
- [self fullGC].
  [totalFreeOldSpace < growHeadroom
  and: [(self growOldSpaceByAtLeast: 0) notNil]] whileTrue:
  [totalFreeOldSpace >= growHeadroom ifTrue:
  [^true]].
  lowSpaceThreshold > totalFreeOldSpace ifTrue: "space is low"
  [lowSpaceThreshold := 0. "avoid signalling low space twice"
  ^false].
  ^true!

Item was added:
+ ----- Method: SpurSegmentManager>>totalOldSpaceSize (in category 'accessing') -----
+ totalOldSpaceSize
+ ^totalHeapSizeIncludingBridges!