VM Maker: VMMaker.oscog-cb.2372.mcz

 
ClementBera uploaded a new version of VMMaker to project VM Maker:
http://source.squeak.org/VMMaker/VMMaker.oscog-cb.2372.mcz

==================== Summary ====================

Name: VMMaker.oscog-cb.2372
Author: cb
Time: 26 April 2018, 9:51:49.730559 am
UUID: c9b30169-cf64-4a87-b0bc-a3bb1cb5e1b4
Ancestors: VMMaker.oscog-VB.2371

Fixed a bug in inlined allocation (or so I think... Did not check the tests)

Added more Simulator side assertions & debug APIs.

Added postSwizzle action, which by default does not nothing, but allow compactors do abuse the swizzle field in segInfo conveniently since it's used at start-up only.

Work on SelectiveCompaction which I believe is research-only compactor (at least for now).

=============== Diff against VMMaker.oscog-VB.2371 ===============

Item was changed:
  ----- Method: CogObjectRepresentationForSpur>>genGetUninitializedInstanceWithClassIndex:numSlots:format:into: (in category 'inline primitive support') -----
  genGetUninitializedInstanceWithClassIndex: classIndex numSlots: numSlots format: format into: destReg
  "Write in destReg the pointer to the object (adjusted based on header size)
  Can deal with large header but not with with old space allocation (max allocation size)"
  | overflowHeader header lowNumSlots |
  <var: #overflowHeader type: #usqLong>
  numSlots >= objectMemory fixedFieldsOfClassFormatMask ifTrue: [^UnimplementedOperation].
  cogit MoveAw: objectMemory freeStartAddress R: destReg.
  numSlots >= objectMemory numSlotsMask
  ifTrue:
  [overflowHeader := numSlots + (objectMemory numSlotsMask << objectMemory numSlotsFullShift).
  self genStoreHeader: overflowHeader intoNewInstance: destReg using: TempReg.
  cogit AddCq: objectMemory baseHeaderSize R: destReg. "Allow to store base header at correct place and to use smallObjectBytes in the rest of the method"
  lowNumSlots := objectMemory numSlotsMask]
  ifFalse:
  [lowNumSlots := numSlots].
  header := objectMemory
  headerForSlots: lowNumSlots
  format: format
  classIndex: classIndex.
  self genStoreHeader: header intoNewInstance: destReg using: TempReg.
  cogit
+ LoadEffectiveAddressMw: (objectMemory objectBytesForSlots: numSlots) r: destReg R: TempReg;
- LoadEffectiveAddressMw: (objectMemory smallObjectBytesForSlots: numSlots) r: destReg R: TempReg;
  MoveR: TempReg Aw: objectMemory freeStartAddress.
 
  !

Item was added:
+ ----- Method: SpurCompactor>>postSwizzleAction (in category 'api') -----
+ postSwizzleAction
+ "Do nothing by default"!

Item was added:
+ ----- Method: SpurMemoryManager>>allOldSpaceFreeChunksDo: (in category 'object enumeration') -----
+ allOldSpaceFreeChunksDo: aBlock
+ <inline: true>
+ <doNotGenerate> "Could be generated, but used for debug only"
+ self allOldSpaceFreeChunksFrom: self firstObject do: aBlock!

Item was added:
+ ----- Method: SpurMemoryManager>>allOldSpaceFreeChunksFrom:do: (in category 'object enumeration') -----
+ allOldSpaceFreeChunksFrom: initialObject do: aBlock
+ <inline: true>
+ <doNotGenerate> "Could be generated, but used for debug only"
+ self allOldSpaceEntitiesFrom: initialObject
+ do: [:objOop|
+ (self isFreeObject: objOop) ifTrue:
+ [aBlock value: objOop]]!

Item was changed:
  ----- Method: SpurMemoryManager>>detachFreeObject: (in category 'free space') -----
  detachFreeObject: freeChunk
  "This is a rare operation, so its efficiency isn't critical.
  Having a valid prev link for tree nodes would help."
  <inline: false>
  | chunkBytes result |
  chunkBytes := self bytesInObject: freeChunk.
  result := self allocateOldSpaceChunkOfExactlyBytes: chunkBytes suchThat: [:f| f = freeChunk].
+ self assert: result = (self startOfObject: freeChunk).
+ "Following is assertion only. Typical problem is that the free structures (tree/list) keep references to detached object somehow"
+ self cCode: '' inSmalltalk:
+ [self allOldSpaceFreeChunksDo:
+ [ :f | self assert: (self isValidFreeObject: f)]].
+ !
- self assert: result = (self startOfObject: freeChunk)!

Item was changed:
  ----- Method: SpurMemoryManager>>initializeObjectMemory: (in category 'initialization') -----
  initializeObjectMemory: bytesToShift
  "Initialize object memory variables at startup time. Assume endOfMemory at al are
  initialised by the image-reading code via setHeapBase:memoryLimit:endOfMemory:.
  endOfMemory is assumed to point to the end of the last object in the image.
  Assume: image reader also initializes the following variables:
  specialObjectsOop
  lastHash"
  <inline: false>
  | freeListObj |
  "Catch mis-initializations leading to bad translations to C"
  self assert: self baseHeaderSize = self baseHeaderSize.
  self assert: (self maxSlotsForAlloc * self wordSize) asInteger > 0.
  self bootstrapping ifFalse:
  [self
  initSegmentBridgeWithBytes: self bridgeSize
  at: endOfMemory - self bridgeSize].
  segmentManager adjustSegmentSwizzlesBy: bytesToShift.
  "image may be at a different address; adjust oops for new location"
  self adjustAllOopsBy: bytesToShift.
  specialObjectsOop := segmentManager swizzleObj: specialObjectsOop.
 
  "heavily used special objects"
  nilObj := self splObj: NilObject.
  falseObj := self splObj: FalseObject.
  trueObj := self splObj: TrueObject.
 
  "In Cog we insist that nil, true & false are next to each other (Cogit generates tighter
  conditional branch code as a result).  In addition, Spur places the free lists and
  class table root page immediately following them."
  self assert: nilObj = oldSpaceStart.
  self assert: falseObj = (self oldSpaceObjectAfter: nilObj).
  self assert: trueObj = (self oldSpaceObjectAfter: falseObj).
  freeListObj := self oldSpaceObjectAfter: trueObj.
  self setHiddenRootsObj: (self oldSpaceObjectAfter: freeListObj).
  markStack := self swizzleObjStackAt: MarkStackRootIndex.
  weaklingStack := self swizzleObjStackAt: WeaklingStackRootIndex.
  mournQueue := self swizzleObjStackAt: MournQueueRootIndex.
  self assert: self validObjStacks.
  self assert: (self isEmptyObjStack: markStack).
  self assert: (self isEmptyObjStack: weaklingStack).
 
  self initializeFreeSpacePostLoad: freeListObj.
  segmentManager collapseSegmentsPostSwizzle.
  self computeFreeSpacePostSwizzle.
+ compactor postSwizzleAction.
  self initializeOldSpaceFirstFree: freeOldSpaceStart. "initializes endOfMemory, freeStart, free space"
  self initializeNewSpaceVariables.
  scavenger initializeRememberedSet.
  segmentManager checkSegments.
  compactor biasForGC.
 
  "These defaults should depend on machine size; e.g. too small on a powerful laptop, too big on a Pi."
  growHeadroom := 16*1024*1024. "headroom when growing"
  shrinkThreshold := 32*1024*1024. "free space before shrinking"
  self setHeapSizeAtPreviousGC.
  heapGrowthToSizeGCRatio := 0.333333. "By default GC after scavenge if heap has grown by a third since the last GC"!

Item was added:
+ ----- Method: SpurMemoryManager>>totalFreeOldSpace (in category 'debug support') -----
+ totalFreeOldSpace
+ <doNotGenerate>
+ ^ totalFreeOldSpace!

Item was changed:
  SpurSweeper subclass: #SpurSelectiveCompactor
+ instanceVariableNames: 'segmentToFill'
+ classVariableNames: 'MaxOccupationForCompaction'
- instanceVariableNames: 'regionsWithOccupationPercentage regionsToReclaim regionToFill'
- classVariableNames: ''
  poolDictionaries: ''
  category: 'VMMaker-SpurMemoryManager'!
 
+ !SpurSelectiveCompactor commentStamp: 'cb 4/19/2018 08:49' prior: 0!
+ SpurSelectiveCompactor compacts memory by selecting the memory segments with the most free space and compacting only those, to limit fragmentation while being really quick to perform. The algorithm is fast mostly because it does not update pointers: they are updated lazily during the next marking phase, so there is no need to read the fields of objects in other memory segments that the one compacted.
- !SpurSelectiveCompactor commentStamp: 'cb 10/1/2017 10:54' prior: 0!
- SpurSelectiveCompactor compacts memory by selecting the memory regions with the most free space and compacting only those, to limit fragmentation while being really quick to perform. The algorithm is fast mostly because it does not update pointers: they are updated lazily during the next marking phase, so there is no need to read the fields of objects in other memory regions that the one compacted.
 
+ The algorithm works as follow. First, a global sweep pass iterates over the memory linearly, changing unmarked objects to free chunks and concatenating free chunks. During the global sweep phase, the segments of the heap are analysed to determine the percentage of occupation. Second, the least occupied segments are compacted by copying the remaining live objects into an entirely free segment, called regionToFill (we detail later in the paragraph where regionToFill comes from) and changing their values to forwarding objects. The rest of each segment being freed is removed from the freeList, it will be entirely freed at the beginning of the next compaction phase. Third, the next marking phase removes all forwarders. Fourth, at the beginning of the next compaction phase the compacted segments from the previous GC can be entirely marked as free space (No need to check anything inside, there were only forwarders that were removed and free chunks not on the free list). One of the freed s
 egment is then selected as the regionToFill, others are just marked as free space. The compaction is effectively partial, compacting only the most critical regions of the heap to limit fragmentation.
- The algorithm works as follow. First, a global sweep pass iterates over the memory linearly, changing unmarked objects to free space. During the global sweep phase, the heap is analysed in the form of variable sized regions and the percentage of occupation of each region is computed. Second, the least occupied regions are compacted by copying the remaining live objects into an entirely free region, called regionToFill (we detail later in the paragraph where regionToFill comes from) and changing their values to forwarding objects. The rest of each region being freed is temporarily marked as used with hidden word objects. Third, the next marking phase removes all forwarders. Fourth, at the beginning of the next compaction phase the compacted regions from the previous GC can be entirely marked as free space (No need to check anything inside, there were only forwarders that were removed and hidden word objects). One of the freed region is then selected as the regionToFill, others are j
 ust marked as free space. The compaction is effectively partial, compacting only the most critical regions of the heap to limit fragmentation.
 
+ Now this works well when biasForGC is true, but when performing a snapshot, the compactor is just total crap (we need to figure out a solution).
- Now this works well when biasForGC is true, but when performing a snapshot, the compactor instead frees the regions held and then calls a more aggressive compactor to decrease the image size.
 
+ segmentToFill <SegInfo> the segment that will be filled through the copying algorithm
- regionsWithOccupationPercentage <> list of regions with the occupation rate
- regionsToReclaim <> regions that will be freed at the beginning of next compaction (After next marking phase)
- biasForGC <> if false this is a compaction for snapshot, else normal GC
- regionToFill <> the region that will be filled through the copying algorithm
 
  !

Item was added:
+ ----- Method: SpurSelectiveCompactor class>>declareCVarsIn: (in category 'translation') -----
+ declareCVarsIn: aCCodeGenerator
+ aCCodeGenerator var: 'segmentToFill' type: #'SpurSegmentInfo *'!

Item was added:
+ ----- Method: SpurSelectiveCompactor class>>initialize (in category 'initialization') -----
+ initialize
+ super initialize.
+ "If the segment is occupied by more than MaxOccupationForCompaction,
+ it's not worth compacting it, whatever the rest of the system looks like.
+ MaxOccupationForCompaction is included in [0;255]."
+ MaxOccupationForCompaction := 150. "Basically if segment is occupied by more than 60%, not worth compacting"!

Item was added:
+ ----- Method: SpurSelectiveCompactor class>>simulatorClass (in category 'simulation') -----
+ simulatorClass
+ ^SpurSelectiveCompactorSimulator!

Item was added:
+ ----- Method: SpurSelectiveCompactor>>allocateSegmentToFill (in category 'freeing') -----
+ allocateSegmentToFill
+ | res |
+ res := manager growOldSpaceByAtLeast: manager growHeadroom.
+ res ifNil: [self error: 'not enough memory for selective compaction'].!

Item was added:
+ ----- Method: SpurSelectiveCompactor>>assertNoSegmentBeingCompacted (in category 'compaction') -----
+ assertNoSegmentBeingCompacted
+
+ "Assertion only - no segment is being claimed at this point"
+ 0 to: manager numSegments - 1 do:
+ [:i| | segInfo |
+ segInfo := self addressOf: (manager segmentManager segments at: i).
+ self deny: (self isSegmentBeingCompacted: segInfo)].
+ !

Item was changed:
  ----- Method: SpurSelectiveCompactor>>compact (in category 'api') -----
  compact
  <inline: #never> "for profiling"
+ self freePastSegmentsAndSetSegmentToFill.
+ self globalSweepAndSegmentOccupationAnalysis.
+ self selectiveCompaction.
- biasForGC
- ifTrue: [self internalCompact]
- ifFalse: ["Should use a compacting algo instead"self internalCompact]
  !

Item was added:
+ ----- Method: SpurSelectiveCompactor>>compactSegment:freeStart: (in category 'compaction') -----
+ compactSegment: segInfo freeStart: initialFreeStart
+ <var: 'segInfo' type: #'SpurSegmentInfo *'>
+ | currentEntity freeStart bytesInObject numSlots bridge |
+ freeStart := initialFreeStart.
+ bridge := manager segmentManager bridgeFor: segInfo.
+ currentEntity := manager objectStartingAt: segInfo segStart.
+ [self oop: currentEntity isLessThan: bridge] whileTrue:
+ [(manager isFreeObject: currentEntity)
+ ifTrue:
+ ["To avoid confusing too much Spur (especially the leak/free checks), we mark the free chunk as a word object."
+ manager detachFreeObject: currentEntity.
+ manager set: currentEntity classIndexTo: manager wordSizeClassIndexPun formatTo: manager wordIndexableFormat]
+ ifFalse:
+ ["Copy the object in segmentToFill and replace it by a forwarder."
+ self assert: (manager isPinned: currentEntity) not.
+ numSlots := manager numSlotsOfAny: currentEntity.
+ bytesInObject := manager bytesInObject: currentEntity.
+ self assert: (manager objectBytesForSlots: numSlots) = (manager bytesInObject: currentEntity).
+ manager mem: freeStart asVoidPointer cp: (manager startOfObject: currentEntity) asVoidPointer y: bytesInObject.
+ self assert: (manager baseHeader: (manager objectStartingAt: freeStart)) = (manager baseHeader: currentEntity).
+ self assert: (manager fetchPointer: numSlots - 1 ofObject: (manager objectStartingAt: freeStart)) = (manager fetchPointer: numSlots - 1 ofObject: currentEntity).
+ manager forward: currentEntity to: (manager objectStartingAt: freeStart).
+ freeStart := freeStart + (manager objectBytesForSlots: numSlots).
+ self assert: (manager isForwarded: currentEntity).
+ self assert: freeStart < (segmentToFill segLimit - manager bridgeSize)].
+ currentEntity := manager objectAfter: currentEntity limit: manager endOfMemory].
+ self assert: currentEntity = bridge.
+ ^ freeStart!

Item was added:
+ ----- Method: SpurSelectiveCompactor>>compactSegmentsToCompact (in category 'compaction') -----
+ compactSegmentsToCompact
+ "Forwards all objects in segments to compact and removes their freechunks"
+ | freeStart |
+ freeStart := segmentToFill segStart.
+ 0 to: manager numSegments - 1 do:
+ [:i| | segInfo |
+ segInfo := self addressOf: (manager segmentManager segments at: i).
+ (self isSegmentBeingCompacted: segInfo)
+ ifTrue: [freeStart := self compactSegment: segInfo freeStart: freeStart ]].
+
+ "Final free chunk in segment to fill..."
+ manager
+ addFreeChunkWithBytes: segmentToFill segSize - manager bridgeSize + segmentToFill segStart - freeStart
+ at: freeStart.
+
+ self postForwardingAction
+ !

Item was added:
+ ----- Method: SpurSelectiveCompactor>>computeSegmentsToCompact (in category 'compaction') -----
+ computeSegmentsToCompact
+ "Compute segments to claim: least occupied.
+ Answers true if at least 1 segment is being compacted."
+
+ | canStillClaim aboutToClaim aboutToClaimSegment atLeastOneSegmentToCompact |
+ atLeastOneSegmentToCompact := false.
+ aboutToClaimSegment := self findNextSegmentToCompact.
+ "Segment to fill is one of the segment compacted last GC.
+ If no segment were compacted last GC, and that there is
+ at least one segment to compact, allocate a new one."
+ aboutToClaimSegment ifNil: [^false].
+ segmentToFill ifNil: [self findOrAllocateSegmentToFill].
+ canStillClaim := segmentToFill segSize - manager bridgeSize.
+ [aboutToClaimSegment ifNil: [^atLeastOneSegmentToCompact].
+ aboutToClaim := aboutToClaimSegment segSize - manager bridgeSize * ((self occupationOf: aboutToClaimSegment) + 1) // 255. "+1 to round up, this is approx"
+ aboutToClaim < canStillClaim ] whileTrue:
+ [self markSegmentAsBeingCompacted: aboutToClaimSegment.
+ atLeastOneSegmentToCompact := true.
+ canStillClaim := canStillClaim - aboutToClaim.
+ aboutToClaimSegment := self findNextSegmentToCompact].
+ ^atLeastOneSegmentToCompact!

Item was added:
+ ----- Method: SpurSelectiveCompactor>>findAndSetSegmentToFill (in category 'freeing') -----
+ findAndSetSegmentToFill
+ 0 to: manager numSegments - 1 do:
+ [:i| | segInfo firstEntity |
+ segInfo := self addressOf: (manager segmentManager segments at: i).
+ firstEntity := manager objectStartingAt: segInfo segStart.
+ ((manager isFreeObject: firstEntity) and: [(manager objectAfter: firstEntity limit: manager endOfMemory) = (manager segmentManager bridgeFor: segInfo)])
+ ifTrue: [segmentToFill := segInfo. manager detachFreeObject: firstEntity. ^0]].
+ !

Item was added:
+ ----- Method: SpurSelectiveCompactor>>findNextSegmentToCompact (in category 'compaction') -----
+ findNextSegmentToCompact
+ "Answers the next segment to compact or nil if none.
+  The next segment to compact:
+ - cannot be segment 0 (Segment 0 has specific objects
+  (nil, true, etc.) and special size computed at start-up
+  that we don't want to deal with)
+ - cannot be be a segment already being compacted.
+ - cannot contain pinned object (since we're in a copying GC)
+ - cannot be entirely empty (no need to block that empty segment until next marking phase)
+ - cannot have a high occupation rate (> MaxOccupationForCompaction)"
+ | leastOccupied leastOccupiedSegment tempOccupied |
+ leastOccupied := 255.
+ 1 to: manager numSegments - 1 do:
+ [:i| | segInfo |
+ segInfo := self addressOf: (manager segmentManager segments at: i).
+ ((self isSegmentBeingCompacted: segInfo) or: [segInfo containsPinned or: [manager segmentManager isEmptySegment: segInfo] ])
+ ifFalse:
+ [(tempOccupied := self occupationOf: segInfo) <= leastOccupied
+ ifTrue: [ leastOccupied := tempOccupied.
+ leastOccupiedSegment := segInfo ]]].
+ leastOccupied > MaxOccupationForCompaction ifTrue: [^nil].
+ ^ leastOccupiedSegment!

Item was added:
+ ----- Method: SpurSelectiveCompactor>>findOrAllocateSegmentToFill (in category 'freeing') -----
+ findOrAllocateSegmentToFill
+ "There was no compacted segments from past GC that we can directly re-use.
+ We need either to find an empty segment or allocate a new one."
+ self findAndSetSegmentToFill.
+ segmentToFill ifNotNil: [^0].
+ "No empty segment. We need to allocate a new one"
+ self allocateSegmentToFill.
+ "We don't know which segment it is that we've just allocated... So we look for it... This is a bit dumb."
+ self findAndSetSegmentToFill.
+ self assert: segmentToFill ~~ nil.
+ !

Item was added:
+ ----- Method: SpurSelectiveCompactor>>freePastSegmentsAndSetSegmentToFill (in category 'freeing') -----
+ freePastSegmentsAndSetSegmentToFill
+ "The first segment being claimed met becomes the segmentToFill. The others are just freed"
+ segmentToFill := nil.
+ 0 to: manager numSegments - 1 do:
+ [:i| | segInfo |
+ segInfo := self addressOf: (manager segmentManager segments at: i).
+ (self isSegmentBeingCompacted: segInfo)
+ ifTrue:
+ [segmentToFill
+ ifNil: [segmentToFill := segInfo]
+ ifNotNil: [self freeSegment: segInfo]]]!

Item was removed:
- ----- Method: SpurSelectiveCompactor>>freeRegionsFromPastCompactionsAndSetRegionToFill (in category 'freeing regions') -----
- freeRegionsFromPastCompactionsAndSetRegionToFill
- "regionsToReclaim is likely a word object with an index and a list of range of regions to reclaim. Sets the regionToFill"!

Item was added:
+ ----- Method: SpurSelectiveCompactor>>freeSegment: (in category 'freeing') -----
+ freeSegment: segInfo
+ <var: 'segInfo' type: #'SpurSegmentInfo *'>
+ manager addFreeChunkWithBytes: segInfo segSize - manager bridgeSize at: segInfo segStart.!

Item was removed:
- ----- Method: SpurSelectiveCompactor>>globalSweepAndRegionAnalysis (in category 'sweep phase') -----
- globalSweepAndRegionAnalysis
- "Iterate over the whole memory, free unmarked objects, figure out regions based on the current size of the heap (regions are variable sized based on an estimate, segments and where objects end) and save them in regionsWithOccupationPercentage"!

Item was added:
+ ----- Method: SpurSelectiveCompactor>>globalSweepAndSegmentOccupationAnalysis (in category 'sweep phase') -----
+ globalSweepAndSegmentOccupationAnalysis
+ self internalGlobalSweepAndSegmentOccupationAnalysis.
+ manager checkFreeSpace: GCModeFull.
+ manager unmarkSurvivingObjectsForCompact.!

Item was removed:
- ----- Method: SpurSelectiveCompactor>>internalCompact (in category 'api') -----
- internalCompact
- self freeRegionsFromPastCompactionsAndSetRegionToFill.
- self globalSweepAndRegionAnalysis.
- self selectiveCompaction.!

Item was added:
+ ----- Method: SpurSelectiveCompactor>>internalGlobalSweepAndSegmentOccupationAnalysis (in category 'sweep phase') -----
+ internalGlobalSweepAndSegmentOccupationAnalysis
+ "Iterate over old space, free unmarked objects, annotate each segment with each occupation"
+ | currentEntity nextBridge start segmentIndex currentUsed currentUnused |
+ currentEntity := manager firstObject.
+ nextBridge := manager segmentManager bridgeAt: 0.
+ segmentIndex := currentUnused := currentUsed := 0.
+ [self oop: currentEntity isLessThan: manager endOfMemory] whileTrue:
+ [currentEntity = nextBridge
+ ifTrue:
+ ["End of segment, set occupation"
+  self
+ setOccupation: (manager segmentManager segments at: segmentIndex)
+ used: currentUsed
+ unused: currentUnused.
+  currentUnused := currentUsed := 0.
+  segmentIndex := segmentIndex + 1.
+  self unmark: currentEntity.
+  nextBridge := manager segmentManager bridgeAt: segmentIndex]
+ ifFalse:
+ ["In-segment, sweep and compute occupation"
+ (self canUseAsFreeSpace: currentEntity)
+ ifTrue:
+ ["bulkFreeChunkFrom: may change a 1 word header
+ object to a double word header object"
+ start := manager startOfObject: currentEntity.
+ self bulkFreeChunkFrom: currentEntity.
+ currentEntity := manager objectStartingAt: start.
+ currentUnused := currentUnused + (manager numSlotsOfAny: currentEntity)]
+ ifFalse:
+ [self unmark: currentEntity.
+ currentUsed := currentUsed + (manager numSlotsOfAny: currentEntity)]].
+ currentEntity := manager objectAfter: currentEntity limit: manager endOfMemory].
+ "set last segment (last bridge = endOfMemory)"
+ self
+ setOccupation: (manager segmentManager segments at: segmentIndex)
+ used: currentUsed
+ unused: currentUnused.!

Item was added:
+ ----- Method: SpurSelectiveCompactor>>isSegmentBeingCompacted: (in category 'segment access') -----
+ isSegmentBeingCompacted: segInfo
+ <var: 'segInfo' type: #'SpurSegmentInfo *'>
+ "Swizzle is abused bit 8 isClaimed bits 0-7 occupation"
+ ^ segInfo swizzle anyMask: 1 << 8!

Item was added:
+ ----- Method: SpurSelectiveCompactor>>markSegmentAsBeingCompacted: (in category 'segment access') -----
+ markSegmentAsBeingCompacted: segInfo
+ <var: 'segInfo' type: #'SpurSegmentInfo *'>
+ "Swizzle is abused bit 8 isClaimed bits 0-7 occupation"
+ segInfo swizzle: (segInfo swizzle bitOr: 1 << 8)!

Item was added:
+ ----- Method: SpurSelectiveCompactor>>occupationOf: (in category 'segment access') -----
+ occupationOf: segment
+ <var: 'segment' type: #'SpurSegmentInfo *'>
+ "Swizzle is abused bit 8 isClaimed bits 0-7 occupation"
+ ^segment swizzle bitAnd: 16rFF!

Item was added:
+ ----- Method: SpurSelectiveCompactor>>postForwardingAction (in category 'compaction') -----
+ postForwardingAction
+ | allFlags |
+ "For now we don't optimize and just follow everything everywhere on stack and in caches, let's see in the profiler if we need to optimize with those cases. My guess is that this is < 100 microSecond"
+ manager followSpecialObjectsOop.
+ allFlags := BecamePointerObjectFlag + BecameActiveClassFlag bitOr: BecameCompiledMethodFlag.
+ manager coInterpreter postBecomeAction: allFlags.
+ manager postBecomeScanClassTable: allFlags.!

Item was added:
+ ----- Method: SpurSelectiveCompactor>>postSwizzleAction (in category 'api') -----
+ postSwizzleAction
+ "Since the compact abuses the swizzle field of segment, it needs to be rest after start-up."
+ 0 to: manager numSegments - 1 do:
+ [:i| | segInfo |
+ segInfo := self addressOf: (manager segmentManager segments at: i).
+ segInfo swizzle: 0 ]!

Item was changed:
  ----- Method: SpurSelectiveCompactor>>selectiveCompaction (in category 'compaction') -----
  selectiveCompaction
+ "Figures out which segments to compact and compact them into segmentToFill"
+ | atLeastOneSegmentToCompact |
+ self assertNoSegmentBeingCompacted.
+ atLeastOneSegmentToCompact := self computeSegmentsToCompact.
+ "If no compaction we don't pay forwarding cost (stack scan, cache scan, etc.)
+ and we don't allocate segmentToFill if none available."
+ atLeastOneSegmentToCompact
+ ifTrue:
+ [self assert: segmentToFill ~~ nil.
+ self compactSegmentsToCompact]
+ ifFalse:
+ [segmentToFill ifNotNil: [self freeSegment: segmentToFill]].
+ manager checkFreeSpace: GCModeFull.!
- "Iterate over regionsWithOccupationPercentage, compact regions from the least occupied to the most occupied, stops when the next region to compact is filled with more than MinFreeForCompaction or when the segment to fill is full"!

Item was added:
+ ----- Method: SpurSelectiveCompactor>>setOccupation:used:unused: (in category 'segment access') -----
+ setOccupation: segment used: used unused: unused
+ <var: 'segment' type: #'SpurSegmentInfo *'>
+ "Swizzle is abused bit 8 isClaimed bits 0-7 occupation
+ Setting occupation resets the claim bit"
+ | occupation |
+ occupation := used * 255 // (used + unused).
+ segment swizzle: occupation!

Item was added:
+ SpurSelectiveCompactor subclass: #SpurSelectiveCompactorSimulator
+ instanceVariableNames: ''
+ classVariableNames: 'Talking'
+ poolDictionaries: ''
+ category: 'VMMaker-SpurMemoryManagerSimulation'!

Item was added:
+ ----- Method: SpurSelectiveCompactorSimulator class>>initialize (in category 'initialization') -----
+ initialize
+ super initialize.
+ Talking := true.!

Item was added:
+ ----- Method: SpurSelectiveCompactorSimulator>>allocateSegmentToFill (in category 'freeing') -----
+ allocateSegmentToFill
+ self talk: 'Allocation of new segment to fill (none available)'.
+ super allocateSegmentToFill!

Item was added:
+ ----- Method: SpurSelectiveCompactorSimulator>>checkOnlyForwardersOrWordNonImm: (in category 'debugging') -----
+ checkOnlyForwardersOrWordNonImm: segInfo
+ "Also check this is iterable until bridge
+ prevEntity is here for debugging only"
+ | currentEntity prevEntity bridge |
+ self talk: 'Checking segment compacted has fwd/Word only ' , (manager segmentManager indexOfSegment: segInfo) printString.
+ bridge := manager segmentManager bridgeFor: segInfo.
+ currentEntity := manager objectStartingAt: segInfo segStart.
+ [self oop: currentEntity isLessThan: bridge] whileTrue:
+ [self assert: ((manager isWordsNonImm: currentEntity) or: [manager isForwarded: currentEntity]).
+ prevEntity := currentEntity.
+ currentEntity := manager objectAfter: currentEntity limit: manager endOfMemory].
+ self assert: currentEntity = bridge.
+ self talkNoCr: ' OK'.!

Item was added:
+ ----- Method: SpurSelectiveCompactorSimulator>>checkSegmentToFillLooksAllRight (in category 'debugging') -----
+ checkSegmentToFillLooksAllRight
+ "Check only 1 free object at the end or none.
+ Check segment is iterable until bridge"
+ | currentEntity bridge |
+ self talk: 'Checking segment to fill iterable/last is free ' , (manager segmentManager indexOfSegment: segmentToFill) printString.
+ bridge := manager segmentManager bridgeFor: segmentToFill.
+ currentEntity := manager objectStartingAt: segmentToFill segStart.
+ [self oop: currentEntity isLessThan: bridge] whileTrue:
+ [(manager isFreeObject: currentEntity)
+ ifTrue: "should be last entity"
+ [self assert: (manager objectAfter: currentEntity limit: manager endOfMemory) = bridge.
+ self assert: (manager isValidFreeObject: currentEntity)].
+ currentEntity := manager objectAfter: currentEntity limit: manager endOfMemory].
+ "End of iteration should end on bridge"
+ self assert: currentEntity = bridge.
+ self talkNoCr: ' OK'.!

Item was added:
+ ----- Method: SpurSelectiveCompactorSimulator>>checkSegmentsIterable (in category 'debugging') -----
+ checkSegmentsIterable
+ "Check only 1 free object at the end or none.
+ Check segment is iterable until bridge"
+ | currentEntity bridge |
+ self talk: 'Checking segments iterable'.
+ 0 to: manager numSegments - 1 do:
+ [:i| | segInfo |
+ segInfo := self addressOf: (manager segmentManager segments at: i).
+ bridge := manager segmentManager bridgeFor: segInfo.
+ currentEntity := manager objectStartingAt: segInfo segStart.
+ [self oop: currentEntity isLessThan: bridge] whileTrue:
+ [currentEntity := manager objectAfter: currentEntity limit: manager endOfMemory].
+ "End of iteration should end on bridge"
+ self assert: currentEntity = bridge.].
+ self talkNoCr: ' OK'.!

Item was added:
+ ----- Method: SpurSelectiveCompactorSimulator>>compact (in category 'api') -----
+ compact
+ self talk: '~~ Start selective compaction ~~'.
+ self tellMeNumSegmentAndTotalFreeSpace.
+ self tellMeWhichSegmentsAreBeingCompacted.
+ super compact.
+ self tellMeNumSegmentAndTotalFreeSpace.
+ self tellMeWhichSegmentsAreBeingCompacted.
+ manager runLeakCheckerForFreeSpace: GCModeFreeSpace.
+ self talk: '~~ Finished selective compaction ~~'.
+ !

Item was added:
+ ----- Method: SpurSelectiveCompactorSimulator>>compactSegment:freeStart: (in category 'compaction') -----
+ compactSegment: segInfo freeStart: initialFreeStart
+ | res |
+ res := super compactSegment: segInfo freeStart: initialFreeStart.
+ self checkOnlyForwardersOrWordNonImm: segInfo.
+ ^ res!

Item was added:
+ ----- Method: SpurSelectiveCompactorSimulator>>compactSegmentsToCompact (in category 'compaction') -----
+ compactSegmentsToCompact
+ self talk: 'filling segment ' , (manager segmentManager indexOfSegment: segmentToFill).
+ super compactSegmentsToCompact.
+ "Now segments to compacts should have been compacted into segment to fill"
+ self checkSegmentToFillLooksAllRight.
+ !

Item was added:
+ ----- Method: SpurSelectiveCompactorSimulator>>computeSegmentsToCompact (in category 'compaction') -----
+ computeSegmentsToCompact
+ | res |
+ res := super computeSegmentsToCompact.
+ res
+ ifTrue: [self talk: 'There is something to compact!!'.
+ self tellMeWhichSegmentsAreBeingCompacted]
+ ifFalse: [self talk: 'Nothing to compact!!'].
+ ^ res!

Item was added:
+ ----- Method: SpurSelectiveCompactorSimulator>>freePastSegmentsAndSetSegmentToFill (in category 'freeing') -----
+ freePastSegmentsAndSetSegmentToFill
+ super freePastSegmentsAndSetSegmentToFill.
+ segmentToFill
+ ifNil: [self talk: 'No segment to fill so far - will allocate one later if anything to compact']
+ ifNotNil: [self talk: 'filling segment ' , (manager segmentManager indexOfSegment: segmentToFill)].
+ !

Item was added:
+ ----- Method: SpurSelectiveCompactorSimulator>>globalSweepAndSegmentOccupationAnalysis (in category 'sweep phase') -----
+ globalSweepAndSegmentOccupationAnalysis
+ super globalSweepAndSegmentOccupationAnalysis.
+ self talk: 'Occupation map: '.
+ 0 to: manager numSegments - 1 do:
+ [:i| | segInfo |
+ segInfo := self addressOf: (manager segmentManager segments at: i).
+ self talkNoCr: (manager segmentManager indexOfSegment: segInfo) printString.
+ self talkNoCr: '->'.
+ self talkNoCr: (self occupationOf: segInfo) printString.
+ self talkNoCr: '('.
+ self talkNoCr: ((self occupationOf: segInfo) * 100 * 100 // 255 / 100) asFloat printString.
+ self talkNoCr: '%);'].
+ self checkSegmentsIterable.
+ !

Item was added:
+ ----- Method: SpurSelectiveCompactorSimulator>>markSegmentAsBeingCompacted: (in category 'compaction') -----
+ markSegmentAsBeingCompacted: segInfo
+ super markSegmentAsBeingCompacted: segInfo.
+ self talk: 'Marking as being compacted segment ' , (manager segmentManager indexOfSegment: segInfo) printString.!

Item was added:
+ ----- Method: SpurSelectiveCompactorSimulator>>talk: (in category 'debugging') -----
+ talk: string
+ Talking ifTrue: [Transcript cr; show: string]!

Item was added:
+ ----- Method: SpurSelectiveCompactorSimulator>>talkNoCr: (in category 'debugging') -----
+ talkNoCr: string
+ Talking ifTrue: [Transcript show: string]!

Item was added:
+ ----- Method: SpurSelectiveCompactorSimulator>>tellMeNumSegmentAndTotalFreeSpace (in category 'debugging') -----
+ tellMeNumSegmentAndTotalFreeSpace
+ self talk: 'numSegments: ' , manager numSegments printString , ', free space: ' , manager totalFreeOldSpace printString.!

Item was added:
+ ----- Method: SpurSelectiveCompactorSimulator>>tellMeWhichSegmentsAreBeingCompacted (in category 'debugging') -----
+ tellMeWhichSegmentsAreBeingCompacted
+ <doNotGenerate>
+ | beingCompacted |
+ beingCompacted := OrderedCollection new.
+ 0 to: manager numSegments - 1 do:
+ [:i| | segInfo |
+ segInfo := self addressOf: (manager segmentManager segments at: i).
+ (self isSegmentBeingCompacted: segInfo) ifTrue: [beingCompacted add: segInfo]].
+ self
+ talk: 'Segment being compacted: ' ,
+ (beingCompacted collect: [:seg |
+ manager segmentManager indexOfSegment: seg ]) asArray printString
+ !

Item was changed:
  SpurCompactor subclass: #SpurSweeper
+ instanceVariableNames: 'biasForGC'
- instanceVariableNames: 'biasForGC aggressiveCompactor'
  classVariableNames: ''
  poolDictionaries: ''
  category: 'VMMaker-SpurMemoryManager'!
 
  !SpurSweeper commentStamp: 'cb 10/2/2017 11:34' prior: 0!
  SpurSweeper is a sweep-only algorithm, setting the compactor to SpurSweeper effectively changes the fullGC to a mark-sweep non-moving algorithm.
 
  SpurSweeper has two main purposes:
  1. SpurSelectiveCompactor includes a sweep algorithm, inherited from SpurSweeper, and SpurSweeper allows to debug it separatedly.
  2. SpurSweeper is a non-moving GC which can be convenient in some cases (For example when accessing objects from C in a concurrent thread called with ThreadedFFI, the C code can access the objects during fullGC since there won't be conflict with object fields being updated while being read).
 
  For snapshots a non-compacting algortihm does not make sense, hence a more aggressive compactor is called instead (see #compact).!

Item was changed:
  ----- Method: SpurSweeper>>compact (in category 'api') -----
  compact
  <inline: #never> "for profiling"
+ self globalSweep!
- biasForGC
- ifTrue: [self globalSweep]
- ifFalse: ["Should use a compacting algo instead"self globalSweep]!