VM Maker: VMMaker.oscog-cb.2564.mcz

 
ClementBera uploaded a new version of VMMaker to project VM Maker:
http://source.squeak.org/VMMaker/VMMaker.oscog-cb.2564.mcz

==================== Summary ====================

Name: VMMaker.oscog-cb.2564
Author: cb
Time: 13 September 2019, 12:33:19.828433 pm
UUID: c39706ec-0a00-4bdf-b4db-8d2e906e173e
Ancestors: VMMaker.oscog-nice.2563

Stop generating forwarder following code for constants and self in methods in #class and for self in methods in #== / #~~. Self in block has still to be followed because self can be a forwarder in blocks without inst var accesses (anyway self == something in block is not super common).

This SimStackEntry>>mayBeAForwarder was tricky to make work, maybe there's a better solution (cmacro for inBlock, force inlining, etc.).

=============== Diff against VMMaker.oscog-nice.2563 ===============

Item was changed:
  ----- Method: CogObjectRepresentation>>genPrimitiveClass (in category 'primitive generators') -----
  genPrimitiveClass
  | reg |
  reg := ReceiverResultReg.
  cogit methodNumArgs > 0 ifTrue:
  [cogit methodNumArgs > 1 ifTrue:
  [^UnimplementedPrimitive].
  cogit genLoadArgAtDepth: 0 into: (reg := Arg0Reg)].
  (self
  genGetClassObjectOf: reg
  into: ReceiverResultReg
  scratchReg: TempReg
+ mayBeAForwarder: reg ~= ReceiverResultReg) = BadRegisterSet ifTrue:
- instRegIsReceiver: reg = ReceiverResultReg) = BadRegisterSet ifTrue:
  [self
  genGetClassObjectOf: reg
  into: ClassReg
  scratchReg: TempReg
+ mayBeAForwarder: reg ~= ReceiverResultReg.
- instRegIsReceiver: reg = ReceiverResultReg.
  cogit MoveR: ClassReg R: ReceiverResultReg].
  cogit genPrimReturn.
  ^UnfailingPrimitive!

Item was added:
+ ----- Method: CogObjectRepresentation>>isUnannotatableConstant: (in category 'testing') -----
+ isUnannotatableConstant: simStackEntry
+ <inline: true>
+ <var: 'simStackEntry' type: #'CogSimStackEntry *'>
+ ^simStackEntry type = SSConstant
+  and: [(objectMemory isImmediate: simStackEntry constant)
+ or: [(self shouldAnnotateObjectReference: simStackEntry constant) not]]!

Item was removed:
- ----- Method: CogObjectRepresentationForSpur>>genGetClassObjectOf:into:scratchReg:instRegIsReceiver: (in category 'compile abstract instructions') -----
- genGetClassObjectOf: instReg into: destReg scratchReg: scratchReg instRegIsReceiver: instRegIsReceiver
- "Fetch the instance's class into destReg.  If the instance is not the receiver and is forwarded, follow forwarding."
- | jumpIsImm jumpNotForwarded loop |
- <var: #jumpIsImm type: #'AbstractInstruction *'>
- <var: #jumpNotForwarded type: #'AbstractInstruction *'>
- <var: #loop type: #'AbstractInstruction *'>
- (instReg = destReg or: [instReg = scratchReg or: [destReg = scratchReg]]) ifTrue:
- [^BadRegisterSet].
- loop := cogit MoveR: instReg R: scratchReg.
- cogit AndCq: objectMemory tagMask R: scratchReg.
- jumpIsImm := cogit JumpNonZero: 0.
- self flag: #endianness.
- "Get least significant half of header word in destReg"
- cogit MoveMw: 0 r: instReg R: scratchReg.
- "mask off class index"
- cogit AndCq: objectMemory classIndexMask R: scratchReg.
- instRegIsReceiver ifFalse:
- ["if it is forwarded..."
- cogit CmpCq: objectMemory isForwardedObjectClassIndexPun R: scratchReg.
- jumpNotForwarded := cogit JumpNonZero: 0.
- "...follow the forwarding pointer and loop to fetch its classIndex"
- cogit MoveMw: objectMemory baseHeaderSize r: instReg R: instReg.
- cogit Jump: loop.
- jumpNotForwarded jmpTarget: cogit Label].
- jumpIsImm jmpTarget:
- (cogit MoveR: scratchReg R: destReg).
- scratchReg = TempReg
- ifTrue:
- [cogit PushR: instReg.
- self genGetClassObjectOfClassIndex: destReg into: instReg scratchReg: TempReg.
- cogit MoveR: instReg R: destReg.
- cogit PopR: instReg]
- ifFalse:
- [self genGetClassObjectOfClassIndex: destReg into: scratchReg scratchReg: TempReg.
- cogit MoveR: scratchReg R: destReg].
- ^0!

Item was added:
+ ----- Method: CogObjectRepresentationForSpur>>genGetClassObjectOf:into:scratchReg:mayBeAForwarder: (in category 'compile abstract instructions') -----
+ genGetClassObjectOf: instReg into: destReg scratchReg: scratchReg mayBeAForwarder: mayBeAForwarder
+ "Fetch the instance's class into destReg.  If the instance is not the receiver and is forwarded, follow forwarding."
+ | jumpIsImm jumpNotForwarded loop |
+ <var: #jumpIsImm type: #'AbstractInstruction *'>
+ <var: #jumpNotForwarded type: #'AbstractInstruction *'>
+ <var: #loop type: #'AbstractInstruction *'>
+ (instReg = destReg or: [instReg = scratchReg or: [destReg = scratchReg]]) ifTrue:
+ [^BadRegisterSet].
+ loop := cogit MoveR: instReg R: scratchReg.
+ cogit AndCq: objectMemory tagMask R: scratchReg.
+ jumpIsImm := cogit JumpNonZero: 0.
+ self flag: #endianness.
+ "Get least significant half of header word in destReg"
+ cogit MoveMw: 0 r: instReg R: scratchReg.
+ "mask off class index"
+ cogit AndCq: objectMemory classIndexMask R: scratchReg.
+ mayBeAForwarder ifTrue:
+ ["if it is forwarded..."
+ cogit CmpCq: objectMemory isForwardedObjectClassIndexPun R: scratchReg.
+ jumpNotForwarded := cogit JumpNonZero: 0.
+ "...follow the forwarding pointer and loop to fetch its classIndex"
+ cogit MoveMw: objectMemory baseHeaderSize r: instReg R: instReg.
+ cogit Jump: loop.
+ jumpNotForwarded jmpTarget: cogit Label].
+ jumpIsImm jmpTarget:
+ (cogit MoveR: scratchReg R: destReg).
+ scratchReg = TempReg
+ ifTrue:
+ [cogit PushR: instReg.
+ self genGetClassObjectOfClassIndex: destReg into: instReg scratchReg: TempReg.
+ cogit MoveR: instReg R: destReg.
+ cogit PopR: instReg]
+ ifFalse:
+ [self genGetClassObjectOfClassIndex: destReg into: scratchReg scratchReg: TempReg.
+ cogit MoveR: scratchReg R: destReg].
+ ^0!

Item was removed:
- ----- Method: CogObjectRepresentationForSqueakV3>>genGetClassObjectOf:into:scratchReg:instRegIsReceiver: (in category 'compile abstract instructions') -----
- genGetClassObjectOf: instReg into: destReg scratchReg: scratchReg instRegIsReceiver: instRegIsReceiver
- "Fetch the instance's class into destReg.  This is almost identical
- to genGetClassFormatOfNonInt:into:scratchReg: but because we
- put the fetch of SmallInteger between the then and the else for
- compact class/non-compact class we cannot easily share code.
- instRegIsReceiver is ignored.  It is for Spur compatibility where
- objects may be forwarded."
- | jumpIsInt jumpCompact jumpGotClass jumpGotClass2 |
- <var: #jumpIsInt type: #'AbstractInstruction *'>
- <var: #jumpCompact type: #'AbstractInstruction *'>
- <var: #jumpGotClass type: #'AbstractInstruction *'>
- <var: #jumpGotClass2 type: #'AbstractInstruction *'>
- cogit MoveR: instReg R: scratchReg.
- cogit AndCq: 1 R: scratchReg.
- jumpIsInt := cogit JumpNonZero: 0.
- "Get header word in scratchReg"
- cogit MoveMw: 0 r: instReg R: scratchReg.
- "Form the byte index of the compact class field"
- cogit LogicalShiftRightCq: (objectMemory compactClassFieldLSB - objectMemory shiftForWord) R: scratchReg.
- cogit AndCq: self compactClassFieldMask << objectMemory shiftForWord R: scratchReg.
- jumpCompact := cogit JumpNonZero: 0.
- cogit MoveMw: objectMemory classFieldOffset r: instReg R: destReg.
- cogit AndCq: AllButTypeMask signedIntFromLong R: destReg.
- jumpGotClass := cogit Jump: 0.
- jumpIsInt jmpTarget: (cogit genMoveConstant: objectMemory classSmallInteger R: destReg).
- jumpGotClass2 := cogit Jump: 0.
- "Don't have to subtract one from the destReg compactClassArray index because of the header word."
- self assert: objectMemory baseHeaderSize = objectMemory wordSize.
- jumpCompact jmpTarget:
- (cogit annotate: (cogit MoveMw: (objectMemory splObj: CompactClasses) r: scratchReg R: destReg)
- objRef: (objectMemory splObj: CompactClasses)).
- jumpGotClass jmpTarget:
- (jumpGotClass2 jmpTarget: cogit Label).
- ^0!

Item was added:
+ ----- Method: CogObjectRepresentationForSqueakV3>>genGetClassObjectOf:into:scratchReg:mayBeAForwarder: (in category 'compile abstract instructions') -----
+ genGetClassObjectOf: instReg into: destReg scratchReg: scratchReg mayBeAForwarder: mayBeAForwarder
+ "Fetch the instance's class into destReg.  This is almost identical
+ to genGetClassFormatOfNonInt:into:scratchReg: but because we
+ put the fetch of SmallInteger between the then and the else for
+ compact class/non-compact class we cannot easily share code.
+ instRegIsReceiver is ignored.  It is for Spur compatibility where
+ objects may be forwarded."
+ | jumpIsInt jumpCompact jumpGotClass jumpGotClass2 |
+ <var: #jumpIsInt type: #'AbstractInstruction *'>
+ <var: #jumpCompact type: #'AbstractInstruction *'>
+ <var: #jumpGotClass type: #'AbstractInstruction *'>
+ <var: #jumpGotClass2 type: #'AbstractInstruction *'>
+ cogit MoveR: instReg R: scratchReg.
+ cogit AndCq: 1 R: scratchReg.
+ jumpIsInt := cogit JumpNonZero: 0.
+ "Get header word in scratchReg"
+ cogit MoveMw: 0 r: instReg R: scratchReg.
+ "Form the byte index of the compact class field"
+ cogit LogicalShiftRightCq: (objectMemory compactClassFieldLSB - objectMemory shiftForWord) R: scratchReg.
+ cogit AndCq: self compactClassFieldMask << objectMemory shiftForWord R: scratchReg.
+ jumpCompact := cogit JumpNonZero: 0.
+ cogit MoveMw: objectMemory classFieldOffset r: instReg R: destReg.
+ cogit AndCq: AllButTypeMask signedIntFromLong R: destReg.
+ jumpGotClass := cogit Jump: 0.
+ jumpIsInt jmpTarget: (cogit genMoveConstant: objectMemory classSmallInteger R: destReg).
+ jumpGotClass2 := cogit Jump: 0.
+ "Don't have to subtract one from the destReg compactClassArray index because of the header word."
+ self assert: objectMemory baseHeaderSize = objectMemory wordSize.
+ jumpCompact jmpTarget:
+ (cogit annotate: (cogit MoveMw: (objectMemory splObj: CompactClasses) r: scratchReg R: destReg)
+ objRef: (objectMemory splObj: CompactClasses)).
+ jumpGotClass jmpTarget:
+ (jumpGotClass2 jmpTarget: cogit Label).
+ ^0!

Item was changed:
  ----- Method: CogObjectRepresentationForSqueakV3>>genGetClassTagOf:into:scratchReg: (in category 'compile abstract instructions') -----
  genGetClassTagOf: instReg into: destReg scratchReg: scratchReg
  "Compatibility with SpurObjectRepresentation/SpurMemoryManager."
  | entryLabel |
  <var: #entryLabel type: #'AbstractInstruction *'>
  cogit AlignmentNops: (objectMemory wordSize max: 8).
  entryLabel := cogit Label.
+ (self genGetClassObjectOf: instReg into: destReg scratchReg: scratchReg mayBeAForwarder: nil) ~= 0 ifTrue:
- (self genGetClassObjectOf: instReg into: destReg scratchReg: scratchReg instRegIsReceiver: nil) ~= 0 ifTrue:
  [self error: 'internal error'].
  ^entryLabel!

Item was added:
+ ----- Method: CogSimStackEntry>>mayBeAForwarder (in category 'comparing') -----
+ mayBeAForwarder
+ "Receiver is not a forwarder, except in blocks with no inst var access.
+ For now we optimize only the case where receiver is accessed in a method."
+ (type == SSRegister and:
+ [cogit isNonForwarderReceiver: register]) ifTrue: [^false].
+ ^ type ~= SSConstant!

Item was added:
+ ----- Method: Cogit>>inBlock (in category 'accessing') -----
+ inBlock
+ <doNotGenerate>
+ ^inBlock!

Item was changed:
  ----- Method: RegisterAllocatingCogit>>genForwardersInlinedIdenticalOrNotIf: (in category 'bytecode generators') -----
  genForwardersInlinedIdenticalOrNotIf: orNot
  | nextPC branchDescriptor unforwardRcvr argReg targetPC
   unforwardArg  rcvrReg postBranchPC retry fixup
+  comparison rcvrConstant argConstant
-  comparison
   needMergeToTarget needMergeToContinue |
  <var: #branchDescriptor type: #'BytecodeDescriptor *'>
  <var: #toContinueLabel type: #'AbstractInstruction *'>
  <var: #toTargetLabel type: #'AbstractInstruction *'>
  <var: #comparison type: #'AbstractInstruction *'>
  <var: #retry type: #'AbstractInstruction *'>
 
  self extractMaybeBranchDescriptorInto: [ :descr :next :postBranch :target |
  branchDescriptor := descr. nextPC := next. postBranchPC := postBranch. targetPC := target ].
 
  "If an operand is an annotable constant, it may be forwarded, so we need to store it into a
  register so the forwarder check can jump back to the comparison after unforwarding the constant.
  However, if one of the operand is an unnanotable constant, does not allocate a register for it
  (machine code will use operations on constants) and does not generate forwarder checks."
+ unforwardRcvr := (self ssValue: 1) mayBeAForwarder.
+ unforwardArg := self ssTop mayBeAForwarder.
+ (unforwardRcvr not and: [unforwardArg not])
+ ifTrue: [^self genVanillaInlinedIdenticalOrNotIf: orNot].
+ self assert: (unforwardArg or: [unforwardRcvr]).
+ "We use reg for non annotable constants to avoid duplicating objRef."
+ rcvrConstant := objectRepresentation isUnannotatableConstant: (self ssValue: 1).
+ argConstant := objectRepresentation isUnannotatableConstant: self ssTop.
- unforwardRcvr := (self ssValue: 1) type ~= SSConstant.
- unforwardArg := self ssTop type ~= SSConstant.
 
  self
+ allocateEqualsEqualsRegistersArgNeedsReg: argConstant not
+ rcvrNeedsReg: rcvrConstant not
- allocateEqualsEqualsRegistersArgNeedsReg: unforwardArg
- rcvrNeedsReg: unforwardRcvr
  into: [ :rcvr :arg | rcvrReg:= rcvr. argReg := arg ].
 
  "If not followed by a branch, resolve to true or false."
  (branchDescriptor isBranchTrue or: [branchDescriptor isBranchFalse]) ifFalse:
  [^self
+ genIdenticalNoBranchArgIsConstant: argConstant
+ rcvrIsConstant: rcvrConstant
- genIdenticalNoBranchArgIsConstant: unforwardArg not
- rcvrIsConstant: unforwardRcvr not
  argReg: argReg
  rcvrReg: rcvrReg
  orNotIf: orNot].
 
  self assert: (unforwardArg or: [unforwardRcvr]).
  self ssPop: 2. "If we had moveAllButTop: 2 volatileSimStackEntriesToRegistersPreserving: we could avoid the extra ssPop:s"
  self moveVolatileSimStackEntriesToRegistersPreserving:
  (self allocatedRegisters bitOr: (argReg = NoReg
  ifTrue: [self registerMaskFor: rcvrReg]
  ifFalse:
  [rcvrReg = NoReg
  ifTrue: [self registerMaskFor: argReg]
  ifFalse: [self registerMaskFor: rcvrReg and: argReg]])).
  retry := self Label.
  self ssPop: -2.
+ self genCmpArgIsConstant: argConstant rcvrIsConstant: rcvrConstant argReg: argReg rcvrReg: rcvrReg.
- self genCmpArgIsConstant: unforwardArg not rcvrIsConstant: unforwardRcvr not argReg: argReg rcvrReg: rcvrReg.
  self ssPop: 2.
 
  (self fixupAt: nextPC) notAFixup "The next instruction is dead.  we can skip it."
  ifTrue:  [deadCode := true]
  ifFalse: [self deny: deadCode]. "push dummy value below"
 
  "self printSimStack; printSimStack: (self fixupAt: postBranchPC) mergeSimStack"
  "If there are merges to be performed on the forward branches we have to execute
  the merge code only along the path requiring that merge, and exactly once."
  needMergeToTarget := self mergeRequiredForJumpTo: targetPC.
  needMergeToContinue := self mergeRequiredForJumpTo: postBranchPC.
  orNot == branchDescriptor isBranchTrue
  ifFalse: "a == b ifTrue: ... or a ~~ b ifFalse: ... jump on equal to target pc"
  [fixup := needMergeToContinue
  ifTrue: [0] "jumps will fall-through to to-continue merge code"
  ifFalse: [self ensureFixupAt: postBranchPC].
  comparison := self JumpZero: (needMergeToTarget
  ifTrue: [0] "comparison will be fixed up to to-target merge code"
  ifFalse: [self ensureFixupAt: targetPC])]
  ifTrue: "a == b ifFalse: ... or a ~~ b ifTrue: ... jump on equal to post-branch pc"
  [fixup := needMergeToTarget
  ifTrue: [0] "jumps will fall-through to to-target merge code"
  ifFalse: [self ensureFixupAt: targetPC].
  comparison := self JumpZero: (needMergeToContinue
  ifTrue: [0] "comparison will be fixed up to to-continue merge code"
  ifFalse: [self ensureFixupAt: postBranchPC])].
 
  "The forwarders check(s) need(s) to jump back to the comparison (retry) if a forwarder is found,
  else jump forward either to the next forwarder check or to the postBranch or branch target (fixup).
  But if there is merge code along a path, the jump must be to the merge code."
  (unforwardArg and: [unforwardRcvr]) ifTrue:
  [objectRepresentation genEnsureOopInRegNotForwarded: argReg scratchReg: TempReg jumpBackTo: retry].
  objectRepresentation
  genEnsureOopInRegNotForwarded: (unforwardRcvr ifTrue: [rcvrReg] ifFalse: [argReg])
  scratchReg: TempReg
  ifForwarder: retry
  ifNotForwarder: fixup.
  "If fixup is zero then the ifNotForwarder path falls through to a Label which is interpreted
  as either to-continue or to-target, depending on orNot == branchDescriptor isBranchTrue."
  orNot == branchDescriptor isBranchTrue
  ifFalse: "a == b ifTrue: ... or a ~~ b ifFalse: ... jump on equal to target pc"
  [needMergeToContinue ifTrue: "fall-through to to-continue merge code"
  [self Jump: (self ensureFixupAt: postBranchPC)].
  needMergeToTarget ifTrue: "fixup comparison to to-target merge code"
  [comparison jmpTarget: self Label.
  self Jump: (self ensureFixupAt: targetPC)]]
  ifTrue: "a == b ifFalse: ... or a ~~ b ifTrue: ... jump on equal to post-branch pc"
  [needMergeToTarget ifTrue: "fall-through to to-target merge code"
  [self Jump: (self ensureFixupAt: targetPC)].
  needMergeToContinue ifTrue: "fixup comparison to to-continue merge code"
  [comparison jmpTarget: self Label.
  self Jump: (self ensureFixupAt: postBranchPC)]].
 
  deadCode ifFalse: "duplicate the merge fixup's top of stack so as to avoid a false confict."
  [self ssPushDesc: ((self fixupAt: nextPC) mergeSimStack at: simStackPtr + 1)].
  ^0!

Item was changed:
  ----- Method: RegisterAllocatingCogit>>genSpecialSelectorClass (in category 'bytecode generators') -----
  genSpecialSelectorClass
  | topReg destReg scratchReg |
  topReg := self allocateRegForStackEntryAt: 0.
  destReg := self allocateRegNotConflictingWith: (self registerMaskFor: topReg).
  scratchReg := self allocateRegNotConflictingWith: (self registerMaskFor: topReg and: destReg).
  self ssTop popToReg: topReg.
  self asserta: (objectRepresentation
  genGetClassObjectOf: topReg
  into: destReg
  scratchReg: scratchReg
+ mayBeAForwarder: self ssTop mayBeAForwarder) ~= BadRegisterSet.
- instRegIsReceiver: false) ~= BadRegisterSet.
  self ssPop: 1; ssPushRegister: destReg.
  ^0!

Item was changed:
  ----- Method: SimpleStackBasedCogit>>genSpecialSelectorClass (in category 'bytecode generators') -----
  genSpecialSelectorClass
  self MoveMw: 0 r: SPReg R: SendNumArgsReg.
  objectRepresentation
  genGetClassObjectOf: SendNumArgsReg
  into: ClassReg
  scratchReg: TempReg
+ mayBeAForwarder: true.
- instRegIsReceiver: false.
  self MoveR: ClassReg Mw: 0 r: SPReg.
  ^0!

Item was changed:
  ----- Method: SistaCogit>>genForwardersInlinedIdenticalOrNotIf: (in category 'bytecode generators') -----
  genForwardersInlinedIdenticalOrNotIf: orNot
  "Override to count inlined branches if followed by a conditional branch.
  We borrow the following conditional branch's counter and when about to
  inline the comparison we decrement the counter (without writing it back)
  and if it trips simply abort the inlining, falling back to the normal send which
  will then continue to the conditional branch which will trip and enter the abort."
  | nextPC postBranchPC targetBytecodePC branchDescriptor counterReg fixup jumpEqual jumpNotEqual
+  counterAddress countTripped unforwardArg unforwardRcvr argReg rcvrReg regMask
+  rcvrConstant argConstant |
-  counterAddress countTripped unforwardArg unforwardRcvr argReg rcvrReg regMask |
  <var: #fixup type: #'BytecodeFixup *'>
  <var: #countTripped type: #'AbstractInstruction *'>
  <var: #label type: #'AbstractInstruction *'>
  <var: #branchDescriptor type: #'BytecodeDescriptor *'>
  <var: #jumpEqual type: #'AbstractInstruction *'>
  <var: #jumpNotEqual type: #'AbstractInstruction *'>
 
  ((coInterpreter isOptimizedMethod: methodObj) or: [needsFrame not]) ifTrue:
  [^super genForwardersInlinedIdenticalOrNotIf: orNot].
 
  regMask := 0.
 
+ unforwardRcvr := (self ssValue: 1) mayBeAForwarder.
+ unforwardArg := self ssTop mayBeAForwarder.
+ (unforwardRcvr not and: [unforwardArg not])
+ ifTrue: [unforwardRcvr := true.
+ "TODO: use genVanilla with profiling counters (not implemented).
+ ^self genVanillaInlinedIdenticalOrNotIf: orNot"].
+ self assert: (unforwardArg or: [unforwardRcvr]).
+ "We use reg for non annotable constants to avoid duplicating objRef."
+ rcvrConstant := objectRepresentation isUnannotatableConstant: (self ssValue: 1).
+ argConstant := objectRepresentation isUnannotatableConstant: self ssTop.
+
  self extractMaybeBranchDescriptorInto: [ :descr :next :postBranch :target |
  branchDescriptor := descr. nextPC := next. postBranchPC := postBranch. targetBytecodePC := target ].
 
- unforwardRcvr := (self ssValue: 1) type ~= SSConstant.
- unforwardArg := self ssTop type ~= SSConstant.
-
  "If an operand is an annotable constant, it may be forwarded, so we need to store it into a
  register so the forwarder check can jump back to the comparison after unforwarding the constant.
  However, if one of the operand is an unnanotable constant, does not allocate a register for it
  (machine code will use operations on constants)."
  self
+ allocateEqualsEqualsRegistersArgNeedsReg: argConstant not
+ rcvrNeedsReg: rcvrConstant not
- allocateEqualsEqualsRegistersArgNeedsReg: unforwardArg
- rcvrNeedsReg: unforwardRcvr
  into: [ :rcvr :arg | rcvrReg:= rcvr. argReg := arg ].
 
  "Only interested in inlining if followed by a conditional branch."
  (branchDescriptor isBranchTrue or: [branchDescriptor isBranchFalse]) ifFalse:
  [^ self
+ genIdenticalNoBranchArgIsConstant: argConstant
+ rcvrIsConstant: rcvrConstant
- genIdenticalNoBranchArgIsConstant: unforwardArg not
- rcvrIsConstant: unforwardRcvr not
  argReg: argReg
  rcvrReg: rcvrReg
  orNotIf: orNot].
 
  "If branching the stack must be flushed for the merge"
  self ssFlushTo: simStackPtr - 2.
 
  unforwardArg ifTrue: [ objectRepresentation genEnsureOopInRegNotForwarded: argReg scratchReg: TempReg ].
  unforwardRcvr ifTrue: [ objectRepresentation genEnsureOopInRegNotForwarded: rcvrReg scratchReg: TempReg ].
 
  regMask := argReg = NoReg
  ifTrue: [self registerMaskFor: rcvrReg]
  ifFalse:
  [rcvrReg = NoReg
  ifTrue: [self registerMaskFor: argReg]
  ifFalse: [self registerMaskFor: rcvrReg and: argReg]].
  counterReg := self allocateRegNotConflictingWith: regMask.
  self
  genExecutionCountLogicInto: [ :cAddress :countTripBranch |
  counterAddress := cAddress.
  countTripped := countTripBranch ]
  counterReg: counterReg.
+
+ self genCmpArgIsConstant: argConstant rcvrIsConstant: rcvrConstant argReg: argReg rcvrReg: rcvrReg.
-
- self assert: (unforwardArg or: [ unforwardRcvr ]).
- self genCmpArgIsConstant: unforwardArg not rcvrIsConstant: unforwardRcvr not argReg: argReg rcvrReg: rcvrReg.
  self ssPop: 2.
 
  orNot == branchDescriptor isBranchTrue "orNot is true for ~~"
  ifFalse:
  [ fixup := (self ensureNonMergeFixupAt: postBranchPC) asUnsignedInteger.
  self JumpZero:  (self ensureNonMergeFixupAt: targetBytecodePC) asUnsignedInteger ]
  ifTrue:
  [ fixup := (self ensureNonMergeFixupAt: targetBytecodePC) asUnsignedInteger.
  self JumpZero: (self ensureNonMergeFixupAt: postBranchPC) asUnsignedInteger ].
 
  self genFallsThroughCountLogicCounterReg: counterReg counterAddress: counterAddress.
  self Jump: fixup.
 
  countTripped jmpTarget: self Label.
 
  "inlined version of #== ignoring the branchDescriptor if the counter trips to have normal state for the optimizer"
  self ssPop: -2.
  self genCmpArgIsConstant: unforwardArg not rcvrIsConstant: unforwardRcvr not argReg: argReg rcvrReg: rcvrReg.
  self ssPop: 2.
 
  "This code necessarily directly falls through the jumpIf: code which pops the top of the stack into TempReg.
  We therefore directly assign the result to TempReg to save one move instruction"
  jumpEqual := orNot ifFalse: [self JumpZero: 0] ifTrue: [self JumpNonZero: 0].
  self genMoveFalseR: TempReg.
  jumpNotEqual := self Jump: 0.
  jumpEqual jmpTarget: (self genMoveTrueR: TempReg).
  jumpNotEqual jmpTarget: self Label.
  self ssPushRegister: TempReg.
 
  (self fixupAt: nextPC) notAFixup ifTrue: [ branchReachedOnlyForCounterTrip := true ].
 
  ^ 0!

Item was changed:
  ----- Method: SistaRegisterAllocatingCogit>>genForwardersInlinedIdenticalOrNotIf: (in category 'bytecode generators') -----
  genForwardersInlinedIdenticalOrNotIf: orNot
  "Override to count inlined branches if followed by a conditional branch.
  We borrow the following conditional branch's counter and when about to
  inline the comparison we decrement the counter (without writing it back)
  and if it trips simply abort the inlining, falling back to the normal send which
  will then continue to the conditional branch which will trip and enter the abort."
  | nextPC postBranchPC targetBytecodePC branchDescriptor counterReg fixup jumpEqual jumpNotEqual
+  counterAddress countTripped unforwardArg unforwardRcvr argReg rcvrReg regMask
+  rcvrConstant argConstant |
-  counterAddress countTripped unforwardArg unforwardRcvr argReg rcvrReg regMask |
  <var: #fixup type: #'BytecodeFixup *'>
  <var: #countTripped type: #'AbstractInstruction *'>
  <var: #label type: #'AbstractInstruction *'>
  <var: #branchDescriptor type: #'BytecodeDescriptor *'>
  <var: #jumpEqual type: #'AbstractInstruction *'>
  <var: #jumpNotEqual type: #'AbstractInstruction *'>
 
  ((coInterpreter isOptimizedMethod: methodObj) or: [needsFrame not]) ifTrue:
  [^super genForwardersInlinedIdenticalOrNotIf: orNot].
 
  regMask := 0.
 
  self extractMaybeBranchDescriptorInto: [ :descr :next :postBranch :target |
  branchDescriptor := descr. nextPC := next. postBranchPC := postBranch. targetBytecodePC := target ].
 
+ unforwardRcvr := (self ssValue: 1) mayBeAForwarder.
+ unforwardArg := self ssTop mayBeAForwarder.
+ (unforwardRcvr not and: [unforwardArg not])
+ ifTrue: [unforwardRcvr := true.
+ "TODO: use genVanilla with profiling counters (not implemented).
+ ^self genVanillaInlinedIdenticalOrNotIf: orNot"].
+ self assert: (unforwardArg or: [unforwardRcvr]).
+ "We use reg for non annotable constants to avoid duplicating objRef."
+ rcvrConstant := objectRepresentation isUnannotatableConstant: (self ssValue: 1).
+ argConstant := objectRepresentation isUnannotatableConstant: self ssTop.
- unforwardRcvr := (self ssValue: 1) type ~= SSConstant.
- unforwardArg := self ssTop type ~= SSConstant.
 
  "If an operand is an annotable constant, it may be forwarded, so we need to store it into a
  register so the forwarder check can jump back to the comparison after unforwarding the constant.
  However, if one of the operand is an unnanotable constant, does not allocate a register for it
  (machine code will use operations on constants)."
  rcvrReg:= argReg := NoReg.
  self
+ allocateEqualsEqualsRegistersArgNeedsReg: argConstant not
+ rcvrNeedsReg: rcvrConstant not
- allocateEqualsEqualsRegistersArgNeedsReg: unforwardArg
- rcvrNeedsReg: unforwardRcvr
  into: [ :rcvr :arg | rcvrReg:= rcvr. argReg := arg ].
 
  argReg ~= NoReg ifTrue: [ regMask := self registerMaskFor: argReg ].
  rcvrReg ~= NoReg ifTrue: [ regMask := regMask bitOr: (self registerMaskFor: rcvrReg) ].
 
  "Only interested in inlining if followed by a conditional branch."
  (branchDescriptor isBranchTrue or: [branchDescriptor isBranchFalse]) ifFalse:
  [^ self
+ genIdenticalNoBranchArgIsConstant: argConstant
+ rcvrIsConstant: rcvrConstant
- genIdenticalNoBranchArgIsConstant: unforwardArg not
- rcvrIsConstant: unforwardRcvr not
  argReg: argReg
  rcvrReg: rcvrReg
  orNotIf: orNot].
 
  unforwardArg ifTrue: [ objectRepresentation genEnsureOopInRegNotForwarded: argReg scratchReg: TempReg ].
  unforwardRcvr ifTrue: [ objectRepresentation genEnsureOopInRegNotForwarded: rcvrReg scratchReg: TempReg ].
 
  counterReg := self allocateRegNotConflictingWith: regMask.
  self
  genExecutionCountLogicInto: [ :cAddress :countTripBranch |
  counterAddress := cAddress.
  countTripped := countTripBranch ]
  counterReg: counterReg.
 
  self assert: (unforwardArg or: [ unforwardRcvr ]).
+ self genCmpArgIsConstant: argConstant rcvrIsConstant: rcvrConstant argReg: argReg rcvrReg: rcvrReg.
- self genCmpArgIsConstant: unforwardArg not rcvrIsConstant: unforwardRcvr not argReg: argReg rcvrReg: rcvrReg.
  self ssPop: 2.
 
  orNot == branchDescriptor isBranchTrue "orNot is true for ~~"
  ifFalse:
  [ fixup := (self ensureNonMergeFixupAt: postBranchPC) asUnsignedInteger.
  self JumpZero:  (self ensureNonMergeFixupAt: targetBytecodePC) asUnsignedInteger ]
  ifTrue:
  [ fixup := (self ensureNonMergeFixupAt: targetBytecodePC) asUnsignedInteger.
  self JumpZero: (self ensureNonMergeFixupAt: postBranchPC) asUnsignedInteger ].
 
  self genFallsThroughCountLogicCounterReg: counterReg counterAddress: counterAddress.
  self Jump: fixup.
 
  countTripped jmpTarget: self Label.
 
  "inlined version of #== ignoring the branchDescriptor if the counter trips to have normal state for the optimizer"
  self ssPop: -2.
  self genCmpArgIsConstant: unforwardArg not rcvrIsConstant: unforwardRcvr not argReg: argReg rcvrReg: rcvrReg.
  self ssPop: 2.
 
  "This code necessarily directly falls through the jumpIf: code which pops the top of the stack into TempReg.
  We therefore directly assign the result to TempReg to save one move instruction"
  jumpEqual := orNot ifFalse: [self JumpZero: 0] ifTrue: [self JumpNonZero: 0].
  self genMoveFalseR: TempReg.
  jumpNotEqual := self Jump: 0.
  jumpEqual jmpTarget: (self genMoveTrueR: TempReg).
  jumpNotEqual jmpTarget: self Label.
  self ssPushRegister: TempReg.
 
  (self fixupAt: nextPC) notAFixup ifTrue: [ branchReachedOnlyForCounterTrip := true ].
 
  ^ 0!

Item was changed:
  ----- Method: StackToRegisterMappingCogit>>genForwardersInlinedIdenticalOrNotIf: (in category 'bytecode generators') -----
  genForwardersInlinedIdenticalOrNotIf: orNot
  | nextPC branchDescriptor unforwardRcvr argReg targetBytecodePC
+  unforwardArg  rcvrReg rcvrConstant argConstant postBranchPC label fixup |
-  unforwardArg  rcvrReg postBranchPC label fixup |
  <var: #branchDescriptor type: #'BytecodeDescriptor *'>
  <var: #label type: #'AbstractInstruction *'>
 
+ unforwardRcvr := (self ssValue: 1) mayBeAForwarder.
+ unforwardArg := self ssTop mayBeAForwarder.
+ (unforwardRcvr not and: [unforwardArg not])
+ ifTrue: [^self genVanillaInlinedIdenticalOrNotIf: orNot].
+ self assert: (unforwardArg or: [unforwardRcvr]).
+ "We use reg for non annotable constants to avoid duplicating objRef."
+ rcvrConstant := objectRepresentation isUnannotatableConstant: (self ssValue: 1).
+ argConstant := objectRepresentation isUnannotatableConstant: self ssTop.
+
  self extractMaybeBranchDescriptorInto: [ :descr :next :postBranch :target |
  branchDescriptor := descr. nextPC := next. postBranchPC := postBranch. targetBytecodePC := target ].
 
- "If an operand is an annotable constant, it may be forwarded, so we need to store it into a
- register so the forwarder check can jump back to the comparison after unforwarding the constant.
- However, if one of the operand is an unnanotable constant, does not allocate a register for it
- (machine code will use operations on constants) and does not generate forwarder checks."
- unforwardRcvr := (self ssValue: 1) type ~= SSConstant.
- unforwardArg := self ssTop type ~= SSConstant.
-
  self
+ allocateEqualsEqualsRegistersArgNeedsReg: argConstant not
+ rcvrNeedsReg: rcvrConstant not
- allocateEqualsEqualsRegistersArgNeedsReg: unforwardArg
- rcvrNeedsReg: unforwardRcvr
  into: [ :rcvr :arg | rcvrReg:= rcvr. argReg := arg ].
 
  "If not followed by a branch, resolve to true or false."
  (branchDescriptor isBranchTrue or: [branchDescriptor isBranchFalse]) ifFalse:
  [^ self
+ genIdenticalNoBranchArgIsConstant: argConstant
+ rcvrIsConstant: rcvrConstant
- genIdenticalNoBranchArgIsConstant: unforwardArg not
- rcvrIsConstant: unforwardRcvr not
  argReg: argReg
  rcvrReg: rcvrReg
  orNotIf: orNot].
 
  "If branching the stack must be flushed for the merge"
  self ssFlushTo: simStackPtr - 2.
 
  label := self Label.
+ self genCmpArgIsConstant: argConstant rcvrIsConstant: rcvrConstant argReg: argReg rcvrReg: rcvrReg.
- self genCmpArgIsConstant: unforwardArg not rcvrIsConstant: unforwardRcvr not argReg: argReg rcvrReg: rcvrReg.
  self ssPop: 2.
 
  "Since there is a following conditional jump bytecode (unless there is deadCode),
  define non-merge fixups and leave the cond bytecode to set the mergeness."
  (self fixupAt: nextPC) notAFixup
  ifTrue: "The next instruction is dead.  we can skip it."
  [deadCode := true.
  self ensureFixupAt: targetBytecodePC.
  self ensureFixupAt: postBranchPC]
  ifFalse:
  [self deny: deadCode]. "push dummy value below"
 
- self assert: (unforwardArg or: [unforwardRcvr]).
  orNot == branchDescriptor isBranchTrue "orNot is true for ~~"
  ifFalse: "a == b ifTrue: ... or a ~~ b ifFalse: ... jump on equal to target pc"
  [fixup := self ensureNonMergeFixupAt: postBranchPC.
  self JumpZero:  (self ensureNonMergeFixupAt: targetBytecodePC)]
  ifTrue: "a == b ifFalse: ... or a ~~ b ifTrue: ... jump on equal to post-branch pc"
  [fixup := self ensureNonMergeFixupAt: targetBytecodePC.
  self JumpZero: (self ensureNonMergeFixupAt: postBranchPC)].
 
  "The forwarders checks need to jump back to the comparison (label) if a forwarder is found, else
  jump forward either to the next forwarder check or to the postBranch or branch target (fixup)."
  (unforwardArg and: [unforwardRcvr]) ifTrue:
  [objectRepresentation genEnsureOopInRegNotForwarded: argReg scratchReg: TempReg jumpBackTo: label].
  objectRepresentation
  genEnsureOopInRegNotForwarded: (unforwardRcvr ifTrue: [rcvrReg] ifFalse: [argReg])
  scratchReg: TempReg
  ifForwarder: label
  ifNotForwarder: fixup.
 
  "Not reached, execution flow has jumped to fixup"
  deadCode ifFalse:
  [self ssPushConstant: objectMemory trueObject]. "dummy value"
  ^0!

Item was changed:
  ----- Method: StackToRegisterMappingCogit>>genInlinedIdenticalOrNotIf: (in category 'bytecode generators') -----
  genInlinedIdenticalOrNotIf: orNot
  "Decompose code generation for #== into a common constant-folding version,
+ followed by a double dispatch through the objectRepresentation to a version
- followed by a double dispatch throguh the objectRepresentation to a version
  that doesn't deal with forwarders and a version that does."
  | primDescriptor result |
  <var: #primDescriptor type: #'BytecodeDescriptor *'>
  primDescriptor := self generatorAt: byte0.
 
+ ((objectRepresentation isUnannotatableConstant: self ssTop)
+ and: [ objectRepresentation isUnannotatableConstant: (self ssValue: 1)]) ifTrue:
- ((self ssTop type == SSConstant)
- and: [(self ssValue: 1) type == SSConstant]) ifTrue:
  [self assert: primDescriptor isMapped not.
  result := (orNot
  ifFalse: [self ssTop constant = (self ssValue: 1) constant]
  ifTrue: [self ssTop constant ~= (self ssValue: 1) constant])
  ifTrue: [objectMemory trueObject]
  ifFalse: [objectMemory falseObject].
  self ssPop: 2.
  ^self ssPushConstant: result].
 
  ^objectRepresentation genInlinedIdenticalOrNotIfGuts: orNot!

Item was changed:
  ----- Method: StackToRegisterMappingCogit>>genSpecialSelectorClass (in category 'bytecode generators') -----
  genSpecialSelectorClass
  | topReg |
  topReg := self ssTop registerOrNone.
  self ssPop: 1.
  (topReg = NoReg or: [topReg = ClassReg])
  ifTrue: [self ssAllocateRequiredReg: (topReg := SendNumArgsReg) and: ClassReg]
  ifFalse: [self ssAllocateRequiredReg: ClassReg].
  self ssPush: 1.
  self ssTop popToReg: topReg.
  objectRepresentation
  genGetClassObjectOf: topReg
  into: ClassReg
  scratchReg: TempReg
+ mayBeAForwarder: self ssTop mayBeAForwarder.
- instRegIsReceiver: false.
  ^self ssPop: 1; ssPushRegister: ClassReg!

Item was added:
+ ----- Method: StackToRegisterMappingCogit>>isNonForwarderReceiver: (in category 'testing') -----
+ isNonForwarderReceiver: reg
+ "Do not inline (inBlock access)"
+ ^self receiverIsInReceiverResultReg
+ and: [inBlock == 0 "Method, rcvr may be forwarder in blocks without inst var access."
+ and: [reg == ReceiverResultReg]]!