VM Maker: VMMaker.oscog-cb.2376.mcz

 
ClementBera uploaded a new version of VMMaker to project VM Maker:
http://source.squeak.org/VMMaker/VMMaker.oscog-cb.2376.mcz

==================== Summary ====================

Name: VMMaker.oscog-cb.2376
Author: cb
Time: 26 April 2018, 2:12:30.237019 pm
UUID: bf5a9a98-571a-4a0e-985e-9e734854535a
Ancestors: VMMaker.oscog-cb.2375

Finally figured out that Slang is not able to do look-ups in compactor classes for some reason, hence only planning compactor was the only one compiling correctly since it defined everything inside...

So I duplicated printTheBogon and postSwizzleAction to all subclasses.

So I removed inheritance of SpurSelectiveCompactor to SpurSweeper, and duplicated used method.

Eliot, if you have any idea why there's no look-up and how to fix it... I'm pretty sure it's just a class-side method somewhere...  So I can remove all this duplication.

=============== Diff against VMMaker.oscog-cb.2375 ===============

Item was changed:
  CogClass subclass: #SpurCompactor
        instanceVariableNames: 'manager scavenger coInterpreter'
        classVariableNames: ''
        poolDictionaries: 'SpurMemoryManagementConstants VMBasicConstants VMSpurObjectRepresentationConstants'
        category: 'VMMaker-SpurMemoryManager'!

+ !SpurCompactor commentStamp: 'cb 4/26/2018 13:53' prior: 0!
- !SpurCompactor commentStamp: 'cb 10/3/2017 10:49' prior: 0!
  Common superclass of all compactors to define apis and simulation variables.

+ IMPORTANT: This defines APIs only, subclassing is prohibited on auxiliary classes (Slang compiler won't find inherited methods).
+
  The full GC in Spur is split in two, the marking phase and the compactor phase. The subclasses of SpurCompactor are implementations of the second phase, so they are called once the marking phase is finished. SpurCompactor is reponsible for:
  - freeing unmarked objects
  - compacting the live old space objects (though each subclass define what it does, some spurCompactor may not compact)
  - unmarking all objects remaining live
  - updating oops directly referred by the VM when they are moved (remapObj:/shouldRemapObj: thingy)

  The main apis are the following:
  - biasForGC/biasForSnapshot: tells the compactor if the GC is performed for snapshots or not, in general we want to compact more aggressively for snapshots to avoid saving large files with many unused space.
  - compact: main API, should free the unmarked object, unmark the objects remaining live and potentially compact the heap
  - remapObj:/shouldRemapObj: => Not really sure what this does, it seems it has to do with updating oops directly referred by the VM when they are moved.


  Instance Variables
        coInterpreter:                          <StackInterpreter>
        compactedCopySpace:             <SpurNewSpaceSpace>
        manager:                                        <SpurMemoryManager>!

Item was changed:
  ----- Method: SpurCompactor>>postSwizzleAction (in category 'api') -----
  postSwizzleAction
+       self subclassResponsibility!
-       "Do nothing by default"!

Item was changed:
  ----- Method: SpurCompactor>>printTheBogons: (in category 'debugging') -----
  printTheBogons: aBogon
+       self subclassResponsibility!
-       <inline: true>
-       coInterpreter
-               print: 'bogon '; printHexnp: aBogon; cr!

Item was added:
+ ----- Method: SpurPigCompactor>>postSwizzleAction (in category 'compatibility') -----
+ postSwizzleAction
+       "do nothing"
+       !

Item was added:
+ ----- Method: SpurPigCompactor>>printTheBogons: (in category 'debug printing') -----
+ printTheBogons: aBogon
+       <inline: true>
+       coInterpreter
+               print: 'bogon '; printHexnp: aBogon; cr!

Item was added:
+ ----- Method: SpurPlanningCompactor>>postSwizzleAction (in category 'compatibility') -----
+ postSwizzleAction
+       "do nothing"
+       !

Item was changed:
+ SpurCompactor subclass: #SpurSelectiveCompactor
+       instanceVariableNames: 'biasForGC segmentToFill'
- SpurSweeper subclass: #SpurSelectiveCompactor
-       instanceVariableNames: 'segmentToFill'
        classVariableNames: 'MaxOccupationForCompaction'
        poolDictionaries: ''
        category: 'VMMaker-SpurMemoryManager'!

+ !SpurSelectiveCompactor commentStamp: 'cb 4/26/2018 13:59' prior: 0!
- !SpurSelectiveCompactor commentStamp: 'cb 4/26/2018 12:53' prior: 0!
  SpurSelectiveCompactor compacts memory by selecting the memory segments with the most free space and compacting only those, to limit fragmentation while being really quick to perform. The algorithm is fast mostly because it does not update pointers: they are updated lazily during the next marking phase, so there is no need to read the fields of objects in other memory segments that the one compacted.

  The algorithm works as follow. First, a global sweep pass iterates over the memory linearly, changing unmarked objects to free chunks and concatenating free chunks. During the global sweep phase, the segments of the heap are analysed to determine the percentage of occupation. Second, the least occupied segments are compacted by copying the remaining live objects into an entirely free segment, called regionToFill (we detail later in the paragraph where regionToFill comes from), changing their values to forwarding objects and marking the free chunks as unavailable (removed from free list and marked as data objects). Third, the next marking phase removes all forwarders. Fourth, at the beginning of the next compaction phase the compacted segments from the previous GC can be entirely marked as free space (No need to check anything inside, there were only forwarders and trash data). One of the compacted segment is then selected as the segmentToFill, others are just marked as free chunks.


  The compaction is effectively partial, compacting only the most critical segments of the heap to limit fragmentation. Compaction time is crazy low, since a low number of objects are moved and pointer updated is lazily done during the next marking phase, while still preventing memory fragmentation.

  Now this works well when biasForGC is true, but when performing a snapshot, the compactor is just total crap (we need to figure out a solution).

+ IMPORTANT: I could not figure out to make inheritance work so I copied methods from SpurSweeper here.
+
  segmentToFill <SegInfo> the segment that will be filled through the copying algorithm

  !

Item was changed:
  ----- Method: SpurSelectiveCompactor>>assertNoSegmentBeingCompacted (in category 'compaction') -----
  assertNoSegmentBeingCompacted
-
        "Assertion only - no segment is being claimed at this point"
+       | segInfo |
+       <var: 'segInfo' type: #'SpurSegmentInfo *'>
        0 to: manager numSegments - 1 do:
+               [:i|
-               [:i| | segInfo |
                 segInfo := self addressOf: (manager segmentManager segments at: i).
                 self deny: (self isSegmentBeingCompacted: segInfo)].
        !

Item was added:
+ ----- Method: SpurSelectiveCompactor>>biasForGC (in category 'api') -----
+ biasForGC
+       biasForGC := true.!

Item was added:
+ ----- Method: SpurSelectiveCompactor>>biasForSnapshot (in category 'api') -----
+ biasForSnapshot
+       biasForGC := false.!

Item was added:
+ ----- Method: SpurSelectiveCompactor>>bulkFreeChunkFrom: (in category 'sweep phase - copied from SpurSweeper') -----
+ bulkFreeChunkFrom: objOop
+       "ObjOop is either a freeChunk or an object to free, always in old space. The old space entity before objOop is necessarily a marked object.
+        Attempts to free as many byte from objOop, looking ahead for multiple freechunks / objects to free in a row"
+       | bytes start next currentObj |
+       
+       "Avoids pathological case, not point in dealing with non-mergeable free chunks, we would remove them and re-add them to the free list."
+       (self isSingleFreeObject: objOop) ifTrue: [^0].
+       
+       "We free unmarked objects and freechunks next to each others and merge them at the same time"
+       start := manager startOfObject: objOop.
+       currentObj := objOop.
+       bytes := 0.
+       [bytes := bytes + (manager bytesInObject: currentObj).
+       self freeEntity: currentObj.
+       next := manager objectStartingAt: start + bytes.
+       self canUseNextEntityAsFreeSpace: next]
+               whileTrue: [currentObj := next].
+       
+       manager addFreeChunkWithBytes: bytes at: start.
+       
+       ^ next!

Item was added:
+ ----- Method: SpurSelectiveCompactor>>canUseAsFreeSpace: (in category 'sweep phase - copied from SpurSweeper') -----
+ canUseAsFreeSpace: objOop
+       <inline: true>
+       ^ (manager isFreeObject: objOop) or: [(manager isMarked: objOop) not]!

Item was added:
+ ----- Method: SpurSelectiveCompactor>>canUseNextEntityAsFreeSpace: (in category 'sweep phase - copied from SpurSweeper') -----
+ canUseNextEntityAsFreeSpace: next
+       <inline: true>
+       ^ (manager oop: next isLessThan: manager endOfMemory) and: [self canUseAsFreeSpace: next]!

Item was changed:
  ----- Method: SpurSelectiveCompactor>>compactSegmentsToCompact (in category 'compaction') -----
  compactSegmentsToCompact
        "Forwards all objects in segments to compact and removes their freechunks"
+       | segInfo fillStart |
+       <var: 'segInfo' type: #'SpurSegmentInfo *'>
-       | fillStart |
        fillStart := segmentToFill segStart.

         "Removes initial free chunk in segment to fill... (Segment is entirely free)"
        manager detachFreeObject: (manager objectStartingAt: fillStart).

         "Compact each segment to compact..."
        0 to: manager numSegments - 1 do:
+               [:i|
-               [:i| | segInfo |
                 segInfo := self addressOf: (manager segmentManager segments at: i).
                (self isSegmentBeingCompacted: segInfo)
                        ifTrue: [fillStart := self compactSegment: segInfo freeStart: fillStart ]].

         "Final free chunk in segment to fill..."
         manager
                addFreeChunkWithBytes: segmentToFill segSize - manager bridgeSize + segmentToFill segStart - fillStart
                at: fillStart.

         "Follow stack zone and caches..."
        self postForwardingAction
        !

Item was changed:
  ----- Method: SpurSelectiveCompactor>>computeSegmentsToCompact (in category 'compaction') -----
  computeSegmentsToCompact
        "Compute segments to claim: least occupied.
         Answers true if at least 1 segment is being compacted."
-       
        | canStillClaim aboutToClaim aboutToClaimSegment atLeastOneSegmentToCompact |
+       <var: 'aboutToClaimSegment' type: #'SpurSegmentInfo *'>
        atLeastOneSegmentToCompact := false.
        aboutToClaimSegment := self findNextSegmentToCompact.
        "Segment to fill is one of the segment compacted last GC.
         If no segment were compacted last GC, and that there is
         at least one segment to compact, allocate a new one."
        aboutToClaimSegment ifNil: [^false].
        segmentToFill ifNil: [self findOrAllocateSegmentToFill].
        canStillClaim := segmentToFill segSize - manager bridgeSize.
        [aboutToClaimSegment ifNil: [^atLeastOneSegmentToCompact].
         aboutToClaim := aboutToClaimSegment segSize - manager bridgeSize * ((self occupationOf: aboutToClaimSegment) + 1) // 255. "+1 to round up, this is approx"
         aboutToClaim < canStillClaim ] whileTrue:
                [self markSegmentAsBeingCompacted: aboutToClaimSegment.
                 atLeastOneSegmentToCompact := true.
                 canStillClaim := canStillClaim - aboutToClaim.
                 aboutToClaimSegment := self findNextSegmentToCompact].
        ^atLeastOneSegmentToCompact!

Item was changed:
  ----- Method: SpurSelectiveCompactor>>findAndSetSegmentToFill (in category 'freeing') -----
  findAndSetSegmentToFill
+       | segInfo firstEntity |
+       <var: 'segInfo' type: #'SpurSegmentInfo *'>
        0 to: manager numSegments - 1 do:
+               [:i|
-               [:i| | segInfo firstEntity |
                 segInfo := self addressOf: (manager segmentManager segments at: i).
                 firstEntity := manager objectStartingAt: segInfo segStart.
                 ((manager isFreeObject: firstEntity) and: [(manager objectAfter: firstEntity limit: manager endOfMemory) = (manager segmentManager bridgeFor: segInfo)])
                        ifTrue: [segmentToFill := segInfo. ^0]].
        !

Item was changed:
  ----- Method: SpurSelectiveCompactor>>findNextSegmentToCompact (in category 'compaction') -----
  findNextSegmentToCompact
        "Answers the next segment to compact or nil if none.
          The next segment to compact:
         - cannot be segment 0 (Segment 0 has specific objects
          (nil, true, etc.) and special size computed at start-up
          that we don't want to deal with)
         - cannot be be a segment already being compacted.
         - cannot contain pinned object (since we're in a copying GC)
         - cannot be entirely empty (no need to block that empty segment until next marking phase)
         - cannot have a high occupation rate (> MaxOccupationForCompaction)"
+       | leastOccupied leastOccupiedSegment tempOccupied segInfo |
+       <var: 'segInfo' type: #'SpurSegmentInfo *'>
-       | leastOccupied leastOccupiedSegment tempOccupied |
        leastOccupied := 255.
        1 to: manager numSegments - 1 do:
+               [:i|
-               [:i| | segInfo |
                 segInfo := self addressOf: (manager segmentManager segments at: i).
                 ((self isSegmentBeingCompacted: segInfo) or: [segInfo containsPinned or: [manager segmentManager isEmptySegment: segInfo] ])
                        ifFalse:
                                [(tempOccupied := self occupationOf: segInfo) <= leastOccupied
                                        ifTrue: [ leastOccupied := tempOccupied.
                                                         leastOccupiedSegment := segInfo ]]].
        leastOccupied > MaxOccupationForCompaction ifTrue: [^nil].
        ^ leastOccupiedSegment!

Item was added:
+ ----- Method: SpurSelectiveCompactor>>freeEntity: (in category 'sweep phase - copied from SpurSweeper') -----
+ freeEntity: entity
+       <inline: true>
+       (manager isFreeObject: entity)
+               ifFalse: "Freed old space objects are removed from remembered table"
+                       [(manager isRemembered: entity) ifTrue:
+                               [scavenger forgetObject: entity]]
+               ifTrue:  "Merged old space free chunks are removed from free list"
+                       [manager detachFreeObject: entity]
+       !

Item was changed:
  ----- Method: SpurSelectiveCompactor>>freePastSegmentsAndSetSegmentToFill (in category 'freeing') -----
  freePastSegmentsAndSetSegmentToFill   
        "The first segment being claimed met becomes the segmentToFill. The others are just freed."
+       | segInfo |
+       <var: 'segInfo' type: #'SpurSegmentInfo *'>
        segmentToFill := nil.
        0 to: manager numSegments - 1 do:
+               [:i|
-               [:i| | segInfo |
                 segInfo := self addressOf: (manager segmentManager segments at: i).
                 (self isSegmentBeingCompacted: segInfo)
                        ifTrue:
                                [self freeSegment: segInfo.
                                 segmentToFill ifNil: [segmentToFill := segInfo]]]!

Item was added:
+ ----- Method: SpurSelectiveCompactor>>isSingleFreeObject: (in category 'sweep phase - copied from SpurSweeper') -----
+ isSingleFreeObject: objOop
+       <inline: true>
+       | next |
+       ^ (manager isFreeObject: objOop) and:
+               [next := manager objectAfter: objOop limit: manager endOfMemory.
+               (manager oop: next isGreaterThanOrEqualTo: manager endOfMemory) or: [manager isMarked: next]]!

Item was changed:
  ----- Method: SpurSelectiveCompactor>>occupationOf: (in category 'segment access') -----
+ occupationOf: segInfo
+       <var: 'segInfo' type: #'SpurSegmentInfo *'>
- occupationOf: segment
-       <var: 'segment' type: #'SpurSegmentInfo *'>
        "Swizzle is abused bit 8 isClaimed bits 0-7 occupation"
+       ^segInfo swizzle bitAnd: 16rFF!
-       ^segment swizzle bitAnd: 16rFF!

Item was changed:
  ----- Method: SpurSelectiveCompactor>>postSwizzleAction (in category 'api') -----
  postSwizzleAction
        "Since the compact abuses the swizzle field of segment, it needs to be rest after start-up."
+       | segInfo |
+       <var: 'segInfo' type: #'SpurSegmentInfo *'>
        0 to: manager numSegments - 1 do:
+               [:i|
-               [:i| | segInfo |
                 segInfo := self addressOf: (manager segmentManager segments at: i).
                 segInfo swizzle: 0 ]!

Item was added:
+ ----- Method: SpurSelectiveCompactor>>printTheBogons: (in category 'debugging') -----
+ printTheBogons: aBogon
+       <inline: true>
+       coInterpreter
+               print: 'bogon '; printHexnp: aBogon; cr!

Item was added:
+ ----- Method: SpurSelectiveCompactor>>remapObj: (in category 'api') -----
+ remapObj: objOop
+       <api>
+       <inline: false>
+       ^manager vanillaRemapObj: objOop!

Item was changed:
  ----- Method: SpurSelectiveCompactor>>setOccupation:used:unused: (in category 'segment access') -----
+ setOccupation: segInfo used: used unused: unused
+       <var: 'segInfo' type: #'SpurSegmentInfo *'>
- setOccupation: segment used: used unused: unused
-       <var: 'segment' type: #'SpurSegmentInfo *'>
        "Swizzle is abused bit 8 isClaimed bits 0-7 occupation
         Setting occupation resets the claim bit"
        | occupation |
        occupation := used * 255 // (used + unused).
+       segInfo swizzle: occupation!
-       segment swizzle: occupation!

Item was changed:
  ----- Method: SpurSelectiveCompactor>>setOccupationAtIndex:used:unused: (in category 'segment access') -----
  setOccupationAtIndex: segmentIndex used: used unused: unused
        "Swizzle is abused bit 8 isClaimed bits 0-7 occupation
         Setting occupation resets the claim bit"
        | occupation segInfo |
+       <var: 'segInfo' type: #'SpurSegmentInfo *'>
        segInfo := self addressOf: (manager segmentManager segments at: segmentIndex).
        occupation := used * 255 // (used + unused).
        segInfo swizzle: occupation!

Item was added:
+ ----- Method: SpurSelectiveCompactor>>shouldRemapObj: (in category 'api') -----
+ shouldRemapObj: objOop
+       <api>
+       ^manager vanillaShouldRemapObj: objOop!

Item was added:
+ ----- Method: SpurSelectiveCompactor>>unmark: (in category 'sweep phase - copied from SpurSweeper') -----
+ unmark: objOop
+       self assert: ((manager isMarked: objOop) and: [(manager isFreeObject: objOop) not]).
+       (manager isSegmentBridge: objOop) ifFalse: [manager setIsMarkedOf: objOop to: false].
+       (manager isPinned: objOop) ifTrue: [manager segmentManager notePinned: objOop]!

Item was added:
+ ----- Method: SpurSweeper>>postSwizzleAction (in category 'api') -----
+ postSwizzleAction
+       "do nothing"
+       !

Item was added:
+ ----- Method: SpurSweeper>>printTheBogons: (in category 'debugging') -----
+ printTheBogons: aBogon
+       <inline: true>
+       coInterpreter
+               print: 'bogon '; printHexnp: aBogon; cr!