VM Maker: VMMaker.oscog-eem.2854.mcz

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

VM Maker: VMMaker.oscog-eem.2854.mcz

commits-2
 
Eliot Miranda uploaded a new version of VMMaker to project VM Maker:
http://source.squeak.org/VMMaker/VMMaker.oscog-eem.2854.mcz

==================== Summary ====================

Name: VMMaker.oscog-eem.2854
Author: eem
Time: 25 October 2020, 6:14:06.461732 pm
UUID: 44be8c63-c2a2-4810-8280-4c215d9d1cad
Ancestors: VMMaker.oscog-eem.2853

MT Cogit: Implement ceTryLockVMOwner that takes an argument and attempts to lock vmOwner to that argument on IA32 & X64.  Refactor ABI callout marshalling on X64 to use the same CArg0Reg thru CArg3Reg pattern as on ARMv8.  Make CPUID the first processor specific opcode on Intel.

Slang: eliminate a space in function type printing.

=============== Diff against VMMaker.oscog-eem.2853 ===============

Item was changed:
  ----- Method: CoInterpreterMT>>ownVMFromUnidentifiedThread (in category 'vm scheduling') -----
  ownVMFromUnidentifiedThread
  "Attempt to take ownership from a thread that as yet doesn't know its index.
  This supports callbacks where the callback could originate from any thread.
 
  Answer 0 if the owning thread is known to the VM.
  Answer 1 if the owning thread is unknown to the VM and now owns the VM.
  Answer -1 if the owning thread is unknown to the VM and fails to own the VM.
  Answer -2 if the owning thread is unknown to the VM and there is no foreign callback process installed."
  | count threadIndex vmThread |
  <var: #vmThread type: #'CogVMThread *'>
  <inline: false>
  (threadIndex := cogThreadManager ioGetThreadLocalThreadIndex) ~= 0 ifTrue:
  [ "this is a callback from a known thread"
  threadIndex = cogThreadManager getVMOwner ifTrue: "the VM has not been disowned"
  [self assert: (disowningVMThread isNil or: [disowningVMThread = self currentVMThread]).
  disowningVMThread := nil.
  self currentVMThread state: CTMAssignableOrInVM.
  ^VMAlreadyOwnedHenceDoNotDisown].
  ^self ownVM: threadIndex].
  foreignCallbackPriority = 0 ifTrue:
  [^-2].
  count := 0.
  "If the current thread doesn't have an index it's new to the vm
  and we need to allocate a new threadInfo, failing if we can't.
  We also need a process in the foreignCallbackProcessSlot upon
  which to run the thread's eventual callback."
+ [[cogit tryLockVMToIndex: cogThreadManager ioCurrentOSThread] whileFalse:
- [[cogThreadManager tryLockVMToIndex: -1] whileFalse:
  [self waitingPriorityIsAtLeast: foreignCallbackPriority.
  cogThreadManager ioTransferTimeslice].
  (objectMemory splObj: foreignCallbackProcessSlot) ~= objectMemory nilObject] whileFalse:
  [cogThreadManager releaseVM.
  (count := count + 1) > 1000 ifTrue:
  [^-2].
  cogThreadManager ioMilliSleep: 1].
  vmThread := cogThreadManager unusedThreadInfo.
  "N.B.  Keep the VM locked anonymously so that we reserve the non-nil ForeignCallbackProcess
  for this thread, avoiding the race between competing foreign callbacks.  The acquireVMFor: in
  ownVM: will set the vmOwner to the actual index.  So only unlock on failure."
  vmThread isNil ifTrue:
  [cogThreadManager releaseVM.
  ^-1].
+ cogThreadManager setVMOwner: vmThread index.
  vmThread
  state: CTMWantingOwnership;
  priority: foreignCallbackPriority.
  cogThreadManager registerVMThread: vmThread.
  ^self ownVM: vmThread index + OwnVMForeignThreadFlag!

Item was changed:
  ----- Method: CoInterpreterMT>>threadSchedulingLoop: (in category 'vm scheduling') -----
  threadSchedulingLoop: vmThread
  "Enter a loop attempting to run the VM with the highest priority process and
  blocking on the thread's OS semaphore when unable to run that process.
  We will return to this via threadSwitchIfNecessary:from: which is called in the
  middle of transferTo:from: once the active process has been stored in the scheduler."
  <var: #vmThread type: #'CogVMThread *'>
  | attemptToRun |
  <inline: false>
  [self assert: vmThread state = CTMAssignableOrInVM.
  attemptToRun := false.
  (cogThreadManager getVMOwner = vmThread index)
  ifTrue: [attemptToRun := true]
  ifFalse:
+ [(cogit tryLockVMToIndex: vmThread index) ifTrue:
- [(cogThreadManager tryLockVMToIndex: vmThread index) ifTrue:
  ["If relinquishing is true, then primitiveRelinquishProcessor has disowned the
   VM and only a returning call or callback should take ownership in that case."
  relinquishing
  ifTrue: [cogThreadManager releaseVM]
  ifFalse: [attemptToRun := true]]].
  attemptToRun ifTrue:
  [self tryToExecuteSmalltalk: vmThread].
  (cogThreadManager testVMOwnerIs: vmThread index) ifFalse:
  [cogThreadManager waitForWork: vmThread].
  true] whileTrue!

Item was changed:
  ----- Method: CogAbstractInstruction class>>initializeSpecificOpcodes:in: (in category 'class initialization') -----
  initializeSpecificOpcodes: opcodeSymbolSequence in: initializeMethod
  "Declare as class variables, the opcodes in opcodeSymbolSequence.
  Assign values to them from LastRTLOpcode on.  Undeclare any obsolete
  class vars.  The assumption is that initializeMethod defines all class vars
  in the class. This method should be used by subclasses wishing to declare
  their own specific opcodes."
+ ^self initializeSpecificOpcodes: opcodeSymbolSequence in: initializeMethod extraClassVarNames: #()!
- | pool classVariablesDefinedInMethod |
- self assert: self ~~ CogAbstractInstruction.
- pool := initializeMethod methodClass classPool.
- LastRTLCode ifNil:
- [CogRTLOpcodes initialize].
- classVariablesDefinedInMethod := Set new.
- initializeMethod allLiteralsDo:
- [:lit|
- (lit isSymbol and: [pool includesKey: lit])
- ifTrue: [classVariablesDefinedInMethod add: lit]
- ifFalse:
- [(lit isVariableBinding and: [pool includesKey:lit key]) ifTrue:
- [classVariablesDefinedInMethod add: lit key]]].
- "Undeclare any class var not defined in opcodeSymbolSequence or by the method."
- (pool keys reject: [:k| (opcodeSymbolSequence includes: k) or: [classVariablesDefinedInMethod includes: k]]) do:
- [:k|
- Undeclared declare: k from: pool].
- "Declare opcodeSymbolSequence's elements from LastRTLCode on up."
- opcodeSymbolSequence withIndexDo:
- [:classVarName :value|
- pool
- declare: classVarName from: Undeclared;
- at: classVarName put: value + LastRTLCode - 1]!

Item was added:
+ ----- Method: CogAbstractInstruction class>>initializeSpecificOpcodes:in:extraClassVarNames: (in category 'class initialization') -----
+ initializeSpecificOpcodes: opcodeSymbolSequence in: initializeMethod extraClassVarNames: extraClassVarNames
+ "Declare as class variables, the opcodes in opcodeSymbolSequence.
+ Assign values to them from LastRTLOpcode on.  Undeclare any obsolete
+ class vars.  The assumption is that initializeMethod defines all class vars
+ in the class. This method should be used by subclasses wishing to declare
+ their own specific opcodes."
+ | pool classVariablesDefinedInMethod |
+ self assert: self ~~ CogAbstractInstruction.
+ pool := initializeMethod methodClass classPool.
+ LastRTLCode ifNil:
+ [CogRTLOpcodes initialize].
+ classVariablesDefinedInMethod := Set new.
+ initializeMethod allLiteralsDo:
+ [:lit|
+ (lit isSymbol and: [pool includesKey: lit])
+ ifTrue: [classVariablesDefinedInMethod add: lit]
+ ifFalse:
+ [(lit isVariableBinding and: [pool includesKey:lit key]) ifTrue:
+ [classVariablesDefinedInMethod add: lit key]]].
+ "Undeclare any class var not defined in opcodeSymbolSequence or by the method."
+ (pool keys reject: [:k| (opcodeSymbolSequence includes: k)
+ or: [(classVariablesDefinedInMethod includes: k)
+ or: [extraClassVarNames includes: k]]]) do:
+ [:k|
+ Undeclared declare: k from: pool].
+ extraClassVarNames do:
+ [:classVarName|
+ (pool includesKey: classVarName) ifFalse:
+ [pool
+ declare: classVarName from: Undeclared;
+ at: classVarName put: nil]].
+ "Declare opcodeSymbolSequence's elements from LastRTLCode on up."
+ opcodeSymbolSequence withIndexDo:
+ [:classVarName :value|
+ pool
+ declare: classVarName from: Undeclared;
+ at: classVarName put: value + LastRTLCode - 1]!

Item was changed:
  ----- Method: CogAbstractInstruction class>>instVarNamesAndTypesForTranslationDo: (in category 'translation') -----
  instVarNamesAndTypesForTranslationDo: aBinaryBlock
  "Enumerate aBinaryBlock with the names and C type strings for the inst vars to include in an AbstractInstruction struct."
  "((CogAbstractInstruction withAllSubclasses reject:
  [:c|
  (c class lookupSelector: #wordSize) isSubclassResponsibility
  or: [((c lookupSelector: #machineCodeBytes) ifNil: [false] ifNotNil: [:cm| cm isSubclassResponsibility])
  or: [c name includesSubstring: 'ForTests']]])
  sort: [:a :b| a name <= b name]) do:
  [:c| Transcript cr; cr; print: c; cr. c printTypedefOn: Transcript]"
 
+ "The first four fields of an instruction are byte variables. opcode, machineCodeSize, maxSize, annotation.
- "The first four fields of an insruction are byte variables. opcode, machineCodeSize, maxSize, annotation.
  On 64 bits we can get better packing if machine code follows these four..."
+ self filteredInstVarNames do:
- (self wordSize = 8
- ifTrue: [self filteredInstVarNames]
- ifFalse: [(self filteredInstVarNames copyWithout: 'machineCode'), #('machineCode')]) do:
  [:ivn|
  aBinaryBlock
  value: ivn
  value: (ivn caseOf: {
  ['address'] -> [#usqInt]. "usqInt is always large enough to contain a pointer; we do not need to use usqIntptr_t"
  ['machineCode'] -> [self machineCodeDeclaration].
  ['operands'] -> [{#usqInt. '[', NumOperands printString, ']'}].
  ['dependent'] -> ['struct _AbstractInstruction *']}
  otherwise:
  [#'unsigned char'])]!

Item was changed:
  ----- Method: CogAbstractInstruction>>numLowLevelLockOpcodes (in category 'multi-threading') -----
  numLowLevelLockOpcodes
+ <inline: #always>
  self subclassResponsibility!

Item was changed:
  CogAbstractInstruction subclass: #CogIA32Compiler
  instanceVariableNames: ''
+ classVariableNames: 'BSR CDQ CLD CMPXCHGRAw CPUID EAX EBP EBX ECX EDI EDX ESI ESP FSTPD FSTPS IDIVR IMULRR LFENCE LOCK MFENCE MOVSB MOVSD ModReg ModRegInd ModRegIndDisp32 ModRegIndSIB ModRegRegDisp32 ModRegRegDisp8 REP SETE SFENCE SIB1 SIB2 SIB4 SIB8 XCHGRR XMM0L XMM1L XMM2L XMM3L XMM4L XMM5L XMM6L XMM7L'
- classVariableNames: 'BSR CDQ CLD CMPXCHGAwR CMPXCHGMwrR CPUID EAX EBP EBX ECX EDI EDX ESI ESP FSTPD FSTPS IDIVR IMULRR LFENCE LOCK MFENCE MOVSB MOVSD ModReg ModRegInd ModRegIndDisp32 ModRegIndSIB ModRegRegDisp32 ModRegRegDisp8 REP SFENCE SIB1 SIB2 SIB4 SIB8 XCHGAwR XCHGMwrR XCHGRR XMM0L XMM1L XMM2L XMM3L XMM4L XMM5L XMM6L XMM7L'
  poolDictionaries: ''
  category: 'VMMaker-JIT'!
 
  !CogIA32Compiler commentStamp: 'eem 9/14/2015 17:13' prior: 0!
  I generate IA32 (x86) instructions from CogAbstractInstructions.  For reference see
  1. IA-32 Intel® Architecture Software Developer's Manual Volume 2A: Instruction Set Reference, A-M
  2. IA-32 Intel® Architecture Software Developer's Manual Volume 2A: Instruction Set Reference, N-Z
  http://www.intel.com/products/processor/manuals/
  (® is supposed to be the Unicode "registered  sign".
 
  This class does not take any special action to flush the instruction cache on instruction-modification, trusting that Intel and AMD processors correctly invalidate the instruction cache via snooping.  According to the manuals, this will work on systems where code and data have the same virtual address.  The CogICacheFlushingIA32Compiler subclass exists to use the CPUID instruction to serialize instruction-modification for systems with code and data at different virtual addresses.!

Item was added:
+ ----- Method: CogIA32Compiler class>>filteredInstVarNames (in category 'translation') -----
+ filteredInstVarNames
+ "Move the unaligned bytes to the end of the structure."
+ ^(super filteredInstVarNames copyWithout: 'machineCode'), #('machineCode')!

Item was changed:
  ----- Method: CogIA32Compiler class>>initialize (in category 'class initialization') -----
  initialize
  "Initialize various IA32/x86 instruction-related constants.
  [1] IA-32 Intel® Architecture Software Developer's Manual Volume 2A: Instruction Set Reference, A-M"
 
  "CogIA32Compiler initialize"
 
  self ~~ CogIA32Compiler ifTrue: [^self].
 
  "N.B. EAX ECX and EDX are caller-save (scratch) registers.
  EBX ESI and EDI are callee-save; see concreteRegisterFor:"
  EAX := 0.
  ECX := 1.  "Were they completely mad or simply sadistic?"
  EDX := 2.
  EBX := 3.
  ESP := 4.
  EBP := 5.
  ESI := 6.
  EDI := 7.
 
  XMM0L := 0.
  XMM1L := 1.
  XMM2L := 2.
  XMM3L := 3.
  XMM4L := 4.
  XMM5L := 5.
  XMM6L := 6.
  XMM7L := 7.
 
  "Mod R/M Mod fields.  See [1] Sec 2.4, 2.5 & 2.6 & Table 2-2"
  ModRegInd := 0.
  ModRegIndSIB := 4.
  ModRegIndDisp32 := 5.
  ModRegRegDisp8 := 1.
  ModRegRegDisp32 := 2.
  ModReg := 3.
 
  "SIB Scaled Index modes.  See [1] Sec 2.4, 2.5 & 2.6 & Table 2-3"
  SIB1 := 0.
  SIB2 := 1.
  SIB4 := 2.
  SIB8 := 3.
 
  "Specific instructions"
  self
+ initializeSpecificOpcodes: #(CPUID
+ "Arithmetic/Bit count:" CDQ IDIVR IMULRR FSTPS FSTPD CLD REP MOVSB MOVSD BSR XCHGRR
+ "Multi-processing:" LFENCE MFENCE SFENCE LOCK CMPXCHGRAw SETE)
- initializeSpecificOpcodes: #(CDQ IDIVR IMULRR CPUID LFENCE MFENCE SFENCE LOCK CMPXCHGAwR CMPXCHGMwrR XCHGAwR XCHGMwrR XCHGRR FSTPS FSTPD CLD REP MOVSB MOVSD BSR)
  in: thisContext method!

Item was changed:
  ----- Method: CogIA32Compiler>>computeMaximumSize (in category 'generate machine code') -----
(excessive size, no diff calculated)

Item was added:
+ ----- Method: CogIA32Compiler>>concretizeCMPXCHGRAw (in category 'generate machine code - concretize processor-specific') -----
+ concretizeCMPXCHGRAw
+ <inline: true>
+ | addressOperand reg |
+ reg := operands at: 0.
+ addressOperand := operands at: 1.
+ machineCode
+ at: 0 put: 16r0F;
+ at: 1 put: 16rB1;
+ at: 2 put: (self mod: ModRegInd RM: 5 RO: reg);
+ at: 3 put: (addressOperand bitAnd: 16rFF);
+ at: 4 put: (addressOperand >> 8 bitAnd: 16rFF);
+ at: 5 put: (addressOperand >> 16 bitAnd: 16rFF);
+ at: 6 put: (addressOperand >> 24 bitAnd: 16rFF).
+ "cogit processor disassembleInstructionAt: 0 In: machineCode object"
+ ^7!

Item was added:
+ ----- Method: CogIA32Compiler>>concretizeSet: (in category 'generate machine code - concretize processor-specific') -----
+ concretizeSet: conditionCode
+ | reg |
+ reg := operands at: 0.
+ machineCode
+ at: 0 put: 16r0F;
+ at: 1 put: 16r90 + conditionCode;
+ at: 2 put: (self mod: ModReg RM: reg RO: 0).
+ "cogit processor disassembleInstructionAt: 0 In: machineCode object"
+ ^3!

Item was changed:
  ----- Method: CogIA32Compiler>>dispatchConcretize (in category 'generate machine code') -----
  dispatchConcretize
  "Attempt to generate concrete machine code for the instruction at address.
  This is the inner dispatch of concretizeAt: actualAddress which exists only
  to get around the branch size limits in the SqueakV3 (blue book derived)
  bytecode set."
+ opcode >= CPUID ifTrue:
- opcode >= CDQ ifTrue:
  [^self dispatchConcretizeProcessorSpecific].
 
  opcode caseOf: {
  "Noops & Pseudo Ops"
  [Label] -> [^self concretizeLabel].
  [AlignmentNops] -> [^self concretizeAlignmentNops].
  [Fill32] -> [^self concretizeFill32].
  [Nop] -> [^self concretizeNop].
  "Control"
  [Call] -> [^self concretizeCall].
  [CallR] -> [^self concretizeCallR].
  [CallFull] -> [^self concretizeCall].
  [JumpR] -> [^self concretizeJumpR].
  [JumpFull] -> [^self concretizeJumpLong].
  [JumpLong] -> [^self concretizeJumpLong].
  [JumpLongZero] -> [^self concretizeConditionalJump: 16r4].
  [JumpLongNonZero] -> [^self concretizeConditionalJump: 16r5].
  [Jump] -> [^self concretizeJump].
  "Table B-1 Intel® 64 and IA-32 Architectures Software Developer's Manual Volume 1: Basic Architecture"
  [JumpZero] -> [^self concretizeConditionalJump: 16r4].
  [JumpNonZero] -> [^self concretizeConditionalJump: 16r5].
  [JumpNegative] -> [^self concretizeConditionalJump: 16r8].
  [JumpNonNegative] -> [^self concretizeConditionalJump: 16r9].
  [JumpOverflow] -> [^self concretizeConditionalJump: 16r0].
  [JumpNoOverflow] -> [^self concretizeConditionalJump: 16r1].
  [JumpCarry] -> [^self concretizeConditionalJump: 16r2].
  [JumpNoCarry] -> [^self concretizeConditionalJump: 16r3].
  [JumpLess] -> [^self concretizeConditionalJump: 16rC].
  [JumpGreaterOrEqual] -> [^self concretizeConditionalJump: 16rD].
  [JumpGreater] -> [^self concretizeConditionalJump: 16rF].
  [JumpLessOrEqual] -> [^self concretizeConditionalJump: 16rE].
  [JumpBelow] -> [^self concretizeConditionalJump: 16r2].
  [JumpAboveOrEqual] -> [^self concretizeConditionalJump: 16r3].
  [JumpAbove] -> [^self concretizeConditionalJump: 16r7].
  [JumpBelowOrEqual] -> [^self concretizeConditionalJump: 16r6].
  [JumpFPEqual] -> [^self concretizeConditionalJump: 16r4].
  [JumpFPNotEqual] -> [^self concretizeConditionalJump: 16r5].
  [JumpFPLess] -> [^self concretizeConditionalJump: 16r2].
  [JumpFPGreaterOrEqual] -> [^self concretizeConditionalJump: 16r3].
  [JumpFPGreater] -> [^self concretizeConditionalJump: 16r7].
  [JumpFPLessOrEqual] -> [^self concretizeConditionalJump: 16r6].
  [JumpFPOrdered] -> [^self concretizeConditionalJump: 16rB].
  [JumpFPUnordered] -> [^self concretizeConditionalJump: 16rA].
  [RetN] -> [^self concretizeRetN].
  [Stop] -> [^self concretizeStop].
  "Arithmetic"
  [AddCqR] -> [^self concretizeAddCqR].
  [AddCwR] -> [^self concretizeAddCwR].
  [AddRR] -> [^self concretizeOpRR: 16r03].
  [AddcRR] -> [^self concretizeAddcRR].
  [AddcCqR] -> [^self concretizeAddcCqR].
  [AddRdRd] -> [^self concretizeSEE2OpRdRd: 16r58].
  [AddRsRs] -> [^self concretizeSEEOpRsRs: 16r58].
  [AndCqR] -> [^self concretizeAndCqR].
  [AndCwR] -> [^self concretizeAndCwR].
  [AndRR] -> [^self concretizeOpRR: 16r23].
  [TstCqR] -> [^self concretizeTstCqR].
  [CmpCqR] -> [^self concretizeCmpCqR].
  [CmpCwR] -> [^self concretizeCmpCwR].
  [CmpRR] -> [^self concretizeReverseOpRR: 16r39].
  [CmpRdRd] -> [^self concretizeCmpRdRd].
  [CmpRsRs] -> [^self concretizeCmpRsRs].
  [DivRdRd] -> [^self concretizeSEE2OpRdRd: 16r5E].
  [DivRsRs] -> [^self concretizeSEEOpRsRs: 16r5E].
  [MulRdRd] -> [^self concretizeSEE2OpRdRd: 16r59].
  [MulRsRs] -> [^self concretizeSEEOpRsRs: 16r59].
  [OrCqR] -> [^self concretizeOrCqR].
  [OrCwR] -> [^self concretizeOrCwR].
  [OrRR] -> [^self concretizeOpRR: 16r0B].
  [SubCqR] -> [^self concretizeSubCqR].
  [SubCwR] -> [^self concretizeSubCwR].
  [SubRR] -> [^self concretizeOpRR: 16r2B].
  [SubbRR] -> [^self concretizeSubbRR].
  [SubRdRd] -> [^self concretizeSEE2OpRdRd: 16r5C].
  [SubRsRs] -> [^self concretizeSEEOpRsRs: 16r5C].
  [SqrtRd] -> [^self concretizeSqrtRd].
  [SqrtRs] -> [^self concretizeSqrtRs].
  [XorCwR] -> [^self concretizeXorCwR].
  [XorRR] -> [^self concretizeOpRR: 16r33].
  [XorRdRd] -> [^self concretizeXorRdRd].
  [XorRsRs] -> [^self concretizeXorRsRs].
  [NegateR] -> [^self concretizeNegateR].
  [NotR] -> [^self concretizeNotR].
  [LoadEffectiveAddressMwrR] -> [^self concretizeLoadEffectiveAddressMwrR].
  [ArithmeticShiftRightCqR] -> [^self concretizeArithmeticShiftRightCqR].
  [LogicalShiftRightCqR] -> [^self concretizeLogicalShiftRightCqR].
  [LogicalShiftLeftCqR] -> [^self concretizeLogicalShiftLeftCqR].
  [ArithmeticShiftRightRR] -> [^self concretizeArithmeticShiftRightRR].
  [LogicalShiftLeftRR] -> [^self concretizeLogicalShiftLeftRR].
  [ClzRR] -> [^self concretizeClzRR].
  "Data Movement"
  [MoveCqR] -> [^self concretizeMoveCqR].
  [MoveCwR] -> [^self concretizeMoveCwR].
  [MoveRR] -> [^self concretizeReverseOpRR: 16r89].
  [MoveRdRd] -> [^self concretizeMoveRdRd].
  [MoveRsRs] -> [^self concretizeMoveRsRs].
  [MoveAwR] -> [^self concretizeMoveAwR].
  [MoveRAw] -> [^self concretizeMoveRAw].
  [MoveAbR] -> [^self concretizeMoveAbR].
  [MoveRAb] -> [^self concretizeMoveRAb].
  [MoveMbrR] -> [^self concretizeMoveMbrR].
  [MoveRMbr] -> [^self concretizeMoveRMbr].
  [MoveRM8r] -> [^self concretizeMoveRMbr].
  [MoveM8rR] -> [^self concretizeMoveM8rR].
  [MoveM16rR] -> [^self concretizeMoveM16rR].
  [MoveRM16r] -> [^self concretizeMoveRM16r].
  [MoveM32rR] -> [^self concretizeMoveMwrR].
  [MoveRM32r] -> [^self concretizeMoveRMwr].
  [MoveM32rRs] -> [^self concretizeMoveM32rRs].
  [MoveRsM32r] -> [^self concretizeMoveRsM32r].
  [MoveM64rRd] -> [^self concretizeMoveM64rRd].
  [MoveMwrR] -> [^self concretizeMoveMwrR].
  [MoveXbrRR] -> [^self concretizeMoveXbrRR].
  [MoveRXbrR] -> [^self concretizeMoveRXbrR].
  [MoveXwrRR] -> [^self concretizeMoveXwrRR].
  [MoveRXwrR] -> [^self concretizeMoveRXwrR].
  [MoveRMwr] -> [^self concretizeMoveRMwr].
  [MoveRdM64r] -> [^self concretizeMoveRdM64r].
  [PopR] -> [^self concretizePopR].
  [PushR] -> [^self concretizePushR].
  [PushCq] -> [^self concretizePushCq].
  [PushCw] -> [^self concretizePushCw].
  [PrefetchAw] -> [^self concretizePrefetchAw].
  "Conversion"
  [ConvertRRd] -> [^self concretizeConvertRRd].
  [ConvertRdR] -> [^self concretizeConvertRdR].
 
  [ConvertRsRd] -> [^self concretizeConvertRsRd].
  [ConvertRdRs] -> [^self concretizeConvertRdRs].
  [ConvertRsR] -> [^self concretizeConvertRsR].
  [ConvertRRs] -> [^self concretizeConvertRRs].
 
  [SignExtend8RR] -> [^self concretizeSignExtend8RR].
  [SignExtend16RR] -> [^self concretizeSignExtend16RR].
 
  [ZeroExtend8RR] -> [^self concretizeZeroExtend8RR].
  [ZeroExtend16RR] -> [^self concretizeZeroExtend16RR] }.
 
  ^0 "keep Slang happy"!

Item was changed:
  ----- Method: CogIA32Compiler>>dispatchConcretizeProcessorSpecific (in category 'generate machine code') -----
  dispatchConcretizeProcessorSpecific
  "Attempt to generate concrete machine code for the instruction at address.
  This is part of the inner dispatch of concretizeAt: actualAddress which exists only
  to get around the number of literals limits in the SqueakV3 (blue book derived)
  bytecode set."
  opcode caseOf: {
+ [CPUID] -> [^self concretizeCPUID].
+ "Specific Arithmetic"
- "Specific Control/Data Movement"
  [CDQ] -> [^self concretizeCDQ].
  [IDIVR] -> [^self concretizeIDIVR].
  [IMULRR] -> [^self concretizeMulRR].
+ [XCHGRR] -> [^self concretizeXCHGRR].
+ "Specific Control/Data Movement"
- [CPUID] -> [^self concretizeCPUID].
- [FSTPS] -> [^self concretizeFSTPS].
- [FSTPD] -> [^self concretizeFSTPD].
  [REP] -> [^self concretizeREP].
  [CLD] -> [^self concretizeCLD].
  [MOVSB] -> [^self concretizeMOVSB].
  [MOVSD] -> [^self concretizeMOVSD].
  [BSR] -> [^self concretizeBSR].
+ [FSTPS] -> [^self concretizeFSTPS].
+ [FSTPD] -> [^self concretizeFSTPD].
- [XCHGRR] -> [^self concretizeXCHGRR].
  "Multi-processing"
+ [CMPXCHGRAw] -> [^self concretizeCMPXCHGRAw].
+ "[LFENCE] -> [^self concretizeFENCE: 5]."
- "[CMPXCHGAwR] -> [^self concretizeCMPXCHGAwR].
- [CMPXCHGMwrR] -> [^self concretizeCMPXCHGMwrR]."
- [LFENCE] -> [^self concretizeFENCE: 5].
  [MFENCE] -> [^self concretizeFENCE: 6].
  [SFENCE] -> [^self concretizeFENCE: 7].
+ [LOCK] -> [^self concretizeLOCK].
+ [SETE] -> [^self concretizeSet: 4].
+ "[XCHGAwR] -> [^self concretizeXCHGAwR]."
- "[LOCK] -> [^self concretizeLOCK]."
- [XCHGAwR] -> [^self concretizeXCHGAwR].
  "[XCHGMwrR] -> [^self concretizeXCHGMwrR]."
  }.
  ^0!

Item was changed:
  ----- Method: CogIA32Compiler>>generateLowLevelTryLock: (in category 'multi-threading') -----
  generateLowLevelTryLock: vmOwnerLockAddress
  "Generate a function that attempts to lock the vmOwnerLock and answers
  true if it succeeded."
  <inline: true>
+ | valueReg |
  vmOwnerLockAddress = 0 ifTrue:
  [cogit
+ MoveCq: 1 R: ABIResultReg;
- MoveCq: 1 R: EAX;
  RetN: 0.
  ^self].
+ valueReg := cogit availableRegisterOrNoneIn: (ABICallerSavedRegisterMask bitClear: 1 << EAX).
  cogit
+ MoveMw: 4 r: ESP R: valueReg;
+ MoveCq: 0 R: EAX;
+ gen: LOCK;
+ gen: CMPXCHGRAw operand: valueReg operand: vmOwnerLockAddress;
+ gen: SETE operand: ABIResultReg; "a.k.a. EAX"
- MoveCq: 1 R: EAX;
- gen: MFENCE; "make the XCHG globally consistent"
- gen: XCHGAwR operand: vmOwnerLockAddress operand: EAX;
- gen: SFENCE; "make the store globally visible"
- SubCq: 1 R: EAX; "Since we only ever set the lock to 1 or 0, subtracting 1 sets
-   EAX to 0 if the lock was already locked and non-zero if it wasn't."
  RetN: 0!

Item was changed:
+ ----- Method: CogIA32Compiler>>numLowLevelLockOpcodes (in category 'multi-threading') -----
- ----- Method: CogIA32Compiler>>numLowLevelLockOpcodes (in category 'accessing') -----
  numLowLevelLockOpcodes
+ <inline: #always>
+ "ceTryLockVMOwner:
+ movl 4(%esp), %ecx
+ xorl %eax, %eax
+ lock cmpxchgl %ecx, %ds:&vmOwnerLock
+ setz %al
+ ret
+ ceUnlockVMOwner:
+ xorl %eax, %eax
+ movl %eax, %ds:&vmOwnerLock
+ sfence
+ ret"
+ ^10!
- "push $ebx
- movl #0, %eax
- movl 1, $ebx
- mfence
- lock cmpxchg %eax, &vmOwnerLock; # N.B. 2 instructions
- pop $ebx
- jnz locked
- sfence
- movl 1, $eax
- ret
- locked: ; N.B. Requires an instruction
- movl 0, $eax
- ret"
- ^14!

Item was removed:
- ----- Method: CogIA32CompilerTests>>testXCHGAwR (in category 'tests') -----
- testXCHGAwR
- "self new testXCHGAwR"
- | xchgAwR |
- xchgAwR := CogIA32Compiler classPool at: #XCHGAwR.
- CogIA32CompilerForTests registersWithNamesDo:
- [:reg :regname|
- #(16r555555 16rAAAAAA) do:
- [:addr| | inst len |
- inst := self gen: xchgAwR operand: addr operand: reg.
- len := inst concretizeAt: 0.
- self processor
- disassembleInstructionAt: 0
- In: inst machineCode object
- into: [:str :sz| | plainJane herIntended |
- "Convert e.g. '00000000: movl %eax, 0x2(%eax) : 89 40 02' to  'movl %eax, 0x2(%eax)'"
- plainJane := self strip: str.
- herIntended := 'xchgl ', regname, ', 0x', (addr hex allButFirst: 3).
- self assert: herIntended equals: plainJane.
- self assert: len = sz]]]!

Item was removed:
- ----- Method: CogIA32CompilerTests>>testXCHGMwrR (in category 'tests') -----
- testXCHGMwrR
- "self new testXCHGMwrR"
- | xchgMwrR |
- xchgMwrR := CogIA32Compiler classPool at: #XCHGMwrR.
- CogIA32CompilerForTests registersWithNamesDo:
- [:sreg :srname|
- CogIA32CompilerForTests registersWithNamesDo:
- [:dreg :drname|
- ((1 to: 19 by: 3) collect: [:po2| 2 raisedToInteger: po2]) do:
- [:offset| | memory |
- self resetGen.
- self gen: xchgMwrR operand: offset operand: sreg operand: dreg.
- memory := self generateInstructions.
- self processor
- disassembleInstructionAt: 0
- In: memory
- into: [:str :sz| | plainJane herIntended |
- plainJane := self strip: str.
- herIntended := 'xchgl ', drname, ', 0x', (offset hex allButFirst: 3), '(', srname, ')'.
- self assert: herIntended equals: plainJane.
- self assert: memory size = sz]]]]!

Item was changed:
  ----- Method: CogThreadManager>>acquireVMFor: (in category 'public api') -----
  acquireVMFor: threadIndex
  "Attempt to acquire the VM, eventually blocking until it becomes available.
  Spin until the maxWaitingPriority has been updated if it is lower than this thread's priority."
  <returnTypeC: #'CogVMThread *'>
  | vmThread |
  <var: #vmThread type: #'CogVMThread *'>
  self assert: threadIndex = self ioGetThreadLocalThreadIndex.
  vmThread := self vmThreadAt: threadIndex.
  self assert: (vmThread state = CTMUnavailable
  or: [vmThread state = CTMWantingOwnership]).
+ (cogit tryLockVMOwner: threadIndex) ifFalse:
- (self tryLockVMToIndex: threadIndex) ifFalse:
  [vmThread state: CTMWantingOwnership.
+ [cogit tryLockVMToIndex: threadIndex] whileFalse:
- [self tryLockVMToIndex: threadIndex] whileFalse:
  [[coInterpreter getMaxWaitingPriority < vmThread priority] whileTrue:
+ [coInterpreter waitingPriorityIsAtLeast: vmThread priority].
+ vmOwner ~= threadIndex ifTrue:
- [cogit tryLockVMOwner ifTrue:
- [coInterpreter waitingPriorityIsAtLeast: vmThread priority.
- cogit unlockVMOwner]].
- vmOwner ~= threadIndex ifTrue:
  [self ioWaitOnOSSemaphore: (self addressOf: vmThread osSemaphore)]]].
  vmOSThread := vmThread osThread.
  vmThread state: CTMAssignableOrInVM.
  ^vmThread!

Item was changed:
  ----- Method: CogThreadManager>>ensureRunningVMThread: (in category 'public api') -----
  ensureRunningVMThread: vmIsRelinquishing
+ "Called from checkVMOwnershipFromHeartbeat if the VM is unowned.
+ Hence we are in the heartbeat thread.  The race is against that thread
+ owning the VM and against foreign callbacks."
  <returnTypeC: #void>
- | vmThread |
  <var: #vmThread type: #'CogVMThread *'>
+ self willingVMThread ifNotNil:
+ [:vmThread|
+ "If the VM is relinquishing the processor then only schedule a thread if it has work to do."
- (self tryLockVMToIndex: -1) ifFalse:
- ["self cCode: [coInterpreter print: 'ERVT failed to lock'; cr]." ^nil].
- (vmThread := self willingVMThread) ifNotNil:
- ["If the VM is relinquishing the processor then only schedule a thread if it has work to do."
  (vmIsRelinquishing
   and: [vmThread state ~= CTMWantingOwnership]) ifTrue:
+ [^self].
+ (cogit tryLockVMToIndex: vmThread index) ifFalse: "someone beat us to it..."
+ [^self].
- [self releaseVM.
- ^nil].
- self setVMOwner: vmThread index.
  vmOSThread := vmThread osThread.
+ "release the thread from its blocking loop"
  self ioSignalOSSemaphore: (self addressOf: vmThread osSemaphore).
  self ioTransferTimeslice.
  "self cCode: [coInterpreter print: 'ERVT signalled '; printNum: vmThread index; cr]."
+ ^self].
- ^nil].
 
+ "If the VM is relinquishing the processor then only schedule a thread if it has work to do
+ (willingVMThread not nil above.
+ If we have failed to allocate thread storage before there is no point continuing to
- "If the VM is relinquishing the processor then only schedule a thread if it has work to do."
- vmIsRelinquishing ifTrue:
- [self releaseVM.
- ^nil].
-
- "If we have failed to allocate thread storage before there is no point continuing to
  try to do so.  By this time we should have quite a few threads in the pool."
+ (vmIsRelinquishing or: [memoryIsScarce]) ifFalse:
+ [self unusedThreadInfo ifNotNil:
+ [:vmThread|
+ self setVMOwner: vmThread index.
+ (self startThreadForThreadInfo: vmThread) ifFalse:
+ [self releaseVM]]]!
- (memoryIsScarce
- or: [(vmThread := self unusedThreadInfo) isNil]) ifTrue:
- [self releaseVM.
- "self cCode: [coInterpreter print: 'ERVT memory is scarce or no unused thread info or too many threads'; cr]."
- ^nil].
-
- self setVMOwner: vmThread index.
- (self startThreadForThreadInfo: vmThread) ifFalse:
- [self releaseVM]!

Item was changed:
  ----- Method: CogThreadManager>>ioCurrentOSThread (in category 'simulation') -----
  ioCurrentOSThread
  <doNotGenerate>
  "See platforms/<plat>/vm/sqPlatformSpecific.h for the real definition."
+ ^Processor activeProcess identityHash!
- ^Processor activeProcess!

Item was changed:
  ----- Method: CogThreadManager>>testVMOwnerIs: (in category 'locking') -----
  testVMOwnerIs: index
  "Test what the vmOwner is from a process that may not be the current VM owner"
  | retryCount ownerIsIndex |
  self assert: index ~= 0.
  retryCount := 0.
+ [cogit tryLockVMOwner: index] whileFalse:
- [cogit tryLockVMOwner] whileFalse:
  [(retryCount := retryCount + 1) > 10 ifTrue:
  [self ioTransferTimeslice]].
  ownerIsIndex := self getVMOwner = index.
  cogit unlockVMOwner.
  ^ownerIsIndex!

Item was removed:
- ----- Method: CogThreadManager>>tryLockVMToIndex: (in category 'locking') -----
- tryLockVMToIndex: index
- "Attempt to lock the vmOwner to index, answering if successful."
- | vmThread locked |
- <var: #vmThread type: #'CogVMThread *'>
- self assert: index ~= 0.
- cogit tryLockVMOwner ifFalse: [^false].
- self getVMOwner = 0 ifTrue:
- [self setVMOwner: index.
- cogit unlockVMOwner.
- index >= 1 ifTrue:
- [vmThread := self vmThreadAt: index.
- vmOSThread := vmThread osThread.
- self sqLowLevelMFence].
- ^true].
- (locked := index > 0 and: [self getVMOwner = index]) ifTrue:
- [vmThread := self vmThreadAt: index.
- vmOSThread := vmThread osThread.
- self sqLowLevelMFence].
- cogit unlockVMOwner.
- ^locked!

Item was changed:
  CogAbstractInstruction subclass: #CogX64Compiler
  instanceVariableNames: ''
+ classVariableNames: 'BSR CArg0Reg CArg1Reg CArg2Reg CArg3Reg CDQ CLD CMPXCHGRMr CPUID IDIVR IMULRR LFENCE LOCK MFENCE MOVSB MOVSQ ModReg ModRegInd ModRegIndDisp32 ModRegIndSIB ModRegRegDisp32 ModRegRegDisp8 MoveRAwNoVBR R10 R11 R12 R13 R14 R15 R8 R9 RAX RBP RBX RCX RDI RDX REP RSI RSP SETE SFENCE SIB1 SIB2 SIB4 SIB8 SysV XCHGRR XMM0L XMM10L XMM11L XMM12L XMM13L XMM14L XMM15L XMM1L XMM2L XMM3L XMM4L XMM5L XMM6L XMM7L XMM8L XMM9L'
- classVariableNames: 'BSR CDQ CLD CMPXCHGAwR CMPXCHGMwrR CPUID IDIVR IMULRR LFENCE LOCK MFENCE MOVSB MOVSQ ModReg ModRegInd ModRegIndDisp32 ModRegIndSIB ModRegRegDisp32 ModRegRegDisp8 MoveRAwNoVBR R10 R11 R12 R13 R14 R15 R8 R9 RAX RBP RBX RCX RDI RDX REP RSI RSP SFENCE SIB1 SIB2 SIB4 SIB8 SysV XCHGAwR XCHGMwrR XCHGRR XMM0L XMM10L XMM11L XMM12L XMM13L XMM14L XMM15L XMM1L XMM2L XMM3L XMM4L XMM5L XMM6L XMM7L XMM8L XMM9L'
  poolDictionaries: ''
  category: 'VMMaker-JIT'!
 
  !CogX64Compiler commentStamp: 'eem 9/14/2015 17:12' prior: 0!
  I generate x64 (x86-64) instructions from CogAbstractInstructions.  For reference see
  1. IA-32 Intel® Architecture Software Developer's Manual Volume 2A: Instruction Set Reference, A-M
  2. IA-32 Intel® Architecture Software Developer's Manual Volume 2A: Instruction Set Reference, N-Z
  http://www.intel.com/products/processor/manuals/
  or
  AMD64 Architecture Programmer's Manual Volume 3: General-Purpose and System Instructions
  AMD64 Architecture Programmer's Manual Volume 4: 128-bit Media Instructions
  AMD64 Architecture Programmer's Manual Volume 5: 64-bit Media and x87 Floating Point Instructions
  http://developer.amd.com/resources/documentation-articles/developer-guides-manuals/
  (® is supposed to be the Unicode "registered  sign").!

Item was added:
+ ----- Method: CogX64Compiler class>>filteredInstVarNames (in category 'translation') -----
+ filteredInstVarNames
+ "Move the unaligned bytes to the end of the structure."
+ ^(super filteredInstVarNames copyWithout: 'machineCode'), #('machineCode')!

Item was changed:
  ----- Method: CogX64Compiler class>>initialize (in category 'class initialization') -----
  initialize
  "Initialize various x64 instruction-related constants.
  [1] IA-32 Intel® Architecture Software Developer's Manual Volume 2A: Instruction Set Reference, A-M"
 
  "CogX64Compiler initialize"
 
  self ~~ CogX64Compiler ifTrue: [^self].
 
  (InitializationOptions ifNil: [Dictionary new])
  at: #ABI
  ifPresent: [:abi| SysV := abi asUppercase ~= #WIN64 and: [abi asUppercase ~= #'_WIN64']]
  ifAbsent: [SysV := true]. "Default ABI; set to true for SysV, false for WIN64/_WIN64"
 
+ "Initialize most sets of variables that run from 0 to N - 1..."
+ #( "16 integer registers, rcx, rdx, rbx bizarrely inconsistent"
+ (RAX RCX RDX RBX RSP RBP RSI RDI R8 R9 R10 R11 R12 R13 R14 R15)
+ "16 lower half floating point registers"
+ (XMM0L XMM1L XMM2L XMM3L XMM4L XMM5L XMM6L XMM7L XMM8L XMM9L XMM10L XMM11L XMM12L XMM13L XMM14L XMM15L)
+ "Mod R/M Mod fields.  See [1] Sec 2.4, 2.5 & 2.6 & Table 2-2"
+ (ModRegInd ModRegRegDisp8 ModRegRegDisp32 ModReg ModRegIndSIB ModRegIndDisp32)
+ "SIB Scaled Index modes.  See [1] Sec 2.4, 2.5 & 2.6 & Table 2-3"
+ (SIB1 SIB2 SIB4 SIB8)) do:
+ [:classVarNames|
+ classVarNames doWithIndex:
+ [:k :v|
+ self classPool at: k put: v - 1]].
- RAX := 0.
- RCX := 1.  "Were they completely mad or simply sadistic?"
- RDX := 2.
- RBX := 3.
- RSP := 4.
- RBP := 5.
- RSI := 6.
- RDI := 7.
- R8 := 8.
- R9 := 9.
- R10 := 10.
- R11 := 11.
- R12 := 12.
- R13 := 13.
- R14 := 14.
- R15 := 15.
 
- XMM0L := 0.
- XMM1L := 1.
- XMM2L := 2.
- XMM3L := 3.
- XMM4L := 4.
- XMM5L := 5.
- XMM6L := 6.
- XMM7L := 7.
- XMM8L := 8.
- XMM9L := 9.
- XMM10L := 10.
- XMM11L := 11.
- XMM12L := 12.
- XMM13L := 13.
- XMM14L := 14.
- XMM15L := 15.
-
- "Mod R/M Mod fields.  See [1] Sec 2.4, 2.5 & 2.6 & Table 2-2"
- ModRegInd := 0.
- ModRegIndSIB := 4.
- ModRegIndDisp32 := 5.
- ModRegRegDisp8 := 1.
- ModRegRegDisp32 := 2.
- ModReg := 3.
-
- "SIB Scaled Index modes.  See [1] Sec 2.4, 2.5 & 2.6 & Table 2-3"
- SIB1 := 0.
- SIB2 := 1.
- SIB4 := 2.
- SIB8 := 3.
-
  "Specific instructions"
+ self initializeSpecificOpcodes: #(CPUID
+ "Arithmetic:" CDQ IDIVR IMULRR XCHGRR
+ "Bit count/data movement:" CLD REP MOVSB MOVSQ BSR
+ "Multi-processing:" LFENCE MFENCE SFENCE LOCK CMPXCHGRMr SETE MoveRAwNoVBR)
+ in: thisContext method
+ extraClassVarNames: #(CArg0Reg CArg1Reg CArg2Reg CArg3Reg)!
- self
- initializeSpecificOpcodes: #(CDQ IDIVR IMULRR CPUID LFENCE MFENCE SFENCE LOCK CMPXCHGAwR CMPXCHGMwrR XCHGAwR MoveRAwNoVBR XCHGMwrR XCHGRR CLD REP MOVSB MOVSQ BSR)
- in: thisContext method!

Item was changed:
  ----- Method: CogX64Compiler class>>initializeAbstractRegistersSysV (in category 'class initialization') -----
  initializeAbstractRegistersSysV
  "Assign the abstract registers with the identities/indices of the relevant concrete registers."
  "[1] Figure 3.4 Register Usage in
  System V Application Binary Interface
  AMD64 Architecture Processor Supplement"
 
  "N.B. RAX RCX & RDX are caller-save (scratch) registers.  Hence we use RCX for class and RDX for
  receiver/result since these are written in all normal sends."
 
  ABICalleeSavedRegisterMask := self registerMaskFor: RBX and: R12 and: R13 and: R14 and: R15.
  ABICallerSavedRegisterMask := self
  registerMaskFor: RAX
  and: RCX
  and: RDX
  and: RSI
  and: RDI
  and: R8
  and: R9
  and: R10
  and: R11.
 
  TempReg := RAX.
  ClassReg := RCX.
  ReceiverResultReg := RDX.
  SendNumArgsReg := R9.
  SPReg := RSP.
  FPReg := RBP.
  Arg0Reg := RDI. "So as to agree with C ABI arg 0"
  Arg1Reg := RSI. "So as to agree with C ABI arg 1"
  VarBaseReg := RBX. "Must be callee saved"
  "R8 is either RISCTempReg or Extra6Reg depending on subclass."
  Extra0Reg := R10.
  Extra1Reg := R11.
  Extra2Reg := R12.
  Extra3Reg := R13.
  Extra4Reg := R14.
+ Extra5Reg := R15.
+
+ "The SysV ABI has 6 integer argument registers; we use only 4."
+ CArg0Reg := RDI.
+ CArg1Reg := RSI.
+ CArg2Reg := RDX.
+ CArg3Reg := RCX!
- Extra5Reg := R15!

Item was changed:
  ----- Method: CogX64Compiler class>>initializeAbstractRegistersWin64 (in category 'class initialization') -----
  initializeAbstractRegistersWin64
  "Assign the abstract registers with the identities/indices of the relevant concrete registers."
 
  "N.B. Since receiver/result are written in all normal sends,
  it's better to use scratch registers for them (those which are caller-saved).
  In Win64 ABI, this does not leave that many choices:
  - RAX is TempReg (overwritten by result etc...)
  - RCX and RDX are used for first 2 args (see genMarshallNArgs:arg:arg:arg:arg:)
  - it remains R8,R9,R10 & R11 : we choose the first two"
 
  ABICalleeSavedRegisterMask := self registerMaskFor: RBX and: RSI and: RDI and: R12 and: R13 and: R14 and: R15.
  ABICallerSavedRegisterMask := self
  registerMaskFor: RAX
  and: RCX
  and: RDX
  and: R8
  and: R9
  and: R10
  and: R11.
 
  TempReg := RAX.
  ClassReg := R8.
  ReceiverResultReg := R9.
  SendNumArgsReg := R10.
  SPReg := RSP.
  FPReg := RBP.
  Arg0Reg := RCX. "So as to agree with C ABI arg 0"
  Arg1Reg := RDX. "So as to agree with C ABI arg 1"
  VarBaseReg := RBX. "Must be callee saved"
  "R11 is either RISCTempReg or Extra6Reg depending on subclass."
  Extra0Reg := RDI.
  Extra1Reg := RSI.
  Extra2Reg := R12.
  Extra3Reg := R13.
  Extra4Reg := R14.
+ Extra5Reg := R15.
+
+ CArg0Reg := RCX.
+ CArg1Reg := RDX.
+ CArg2Reg := R8.
+ CArg3Reg := R9!
- Extra5Reg := R15!

Item was changed:
  ----- Method: CogX64Compiler>>computeMaximumSize (in category 'generate machine code') -----
(excessive size, no diff calculated)

Item was added:
+ ----- Method: CogX64Compiler>>concretizeCMPXCHGRMr (in category 'generate machine code - concretize processor-specific') -----
+ concretizeCMPXCHGRMr
+ <inline: true>
+ | valueReg addressReg |
+ valueReg := operands at: 0.
+ addressReg := operands at: 1.
+ machineCode
+ "REX.W + 0F B1/r CMPXCHG r64"
+ at: 0 put: (self rexR: valueReg x: 0 b: addressReg);
+ at: 1 put: 16r0f;
+ at: 2 put: 16rB1;
+ at: 3 put: (self mod: ModRegInd RM: addressReg RO: valueReg).
+ "cogit processor disassembleInstructionAt: 0 In: machineCode object"
+ ^4!

Item was added:
+ ----- Method: CogX64Compiler>>concretizeLOCK (in category 'pro') -----
+ concretizeLOCK
+ <inline: true>
+ machineCode at: 0 put: 16rF0.
+ ^1!

Item was added:
+ ----- Method: CogX64Compiler>>concretizeSet: (in category 'proc') -----
+ concretizeSet: conditionCode
+ | reg offset |
+ offset := (reg := operands at: 0) >= R8
+ ifTrue: [machineCode at: 0 put: 16r40.
+ 1]
+ ifFalse: [0].
+ machineCode
+ at: 0 + offset put: 16r0F;
+ at: 1 + offset put: 16r90 + conditionCode;
+ at: 2 + offset put: (self mod: ModReg RM: (reg bitAnd: 7) RO: 0).
+ "cogit processor disassembleInstructionAt: 0 In: machineCode object"
+ ^3 + offset!

Item was changed:
  ----- Method: CogX64Compiler>>dispatchConcretize (in category 'generate machine code') -----
  dispatchConcretize
  "Attempt to generate concrete machine code for the instruction at address.
  This is the inner dispatch of concretizeAt: actualAddress which exists only
  to get around the branch size limits in the SqueakV3 (blue book derived)
  bytecode set."
+ opcode >= CPUID ifTrue:
- opcode >= CDQ ifTrue:
  [^self dispatchConcretizeProcessorSpecific].
 
  opcode caseOf: {
  "Noops & Pseudo Ops"
  [Label] -> [^self concretizeLabel].
  [AlignmentNops] -> [^self concretizeAlignmentNops].
  [Fill32] -> [^self concretizeFill32].
  [Nop] -> [^self concretizeNop].
  "Control"
  [Call] -> [^self concretizeCall].
  [CallR] -> [^self concretizeCallR].
  [CallFull] -> [^self concretizeCallFull].
  [JumpR] -> [^self concretizeJumpR].
  [JumpFull] -> [^self concretizeJumpFull].
  [JumpLong] -> [^self concretizeJumpLong].
  [JumpLongZero] -> [^self concretizeConditionalJump: 16r4].
  [JumpLongNonZero] -> [^self concretizeConditionalJump: 16r5].
  [Jump] -> [^self concretizeJump].
  "Table B-1 Intel 64 and IA-32 Architectures Software Developer's Manual Volume 1: Basic Architecture"
  [JumpZero] -> [^self concretizeConditionalJump: 16r4].
  [JumpNonZero] -> [^self concretizeConditionalJump: 16r5].
  [JumpNegative] -> [^self concretizeConditionalJump: 16r8].
  [JumpNonNegative] -> [^self concretizeConditionalJump: 16r9].
  [JumpOverflow] -> [^self concretizeConditionalJump: 16r0].
  [JumpNoOverflow] -> [^self concretizeConditionalJump: 16r1].
  [JumpCarry] -> [^self concretizeConditionalJump: 16r2].
  [JumpNoCarry] -> [^self concretizeConditionalJump: 16r3].
  [JumpLess] -> [^self concretizeConditionalJump: 16rC].
  [JumpGreaterOrEqual] -> [^self concretizeConditionalJump: 16rD].
  [JumpGreater] -> [^self concretizeConditionalJump: 16rF].
  [JumpLessOrEqual] -> [^self concretizeConditionalJump: 16rE].
  [JumpBelow] -> [^self concretizeConditionalJump: 16r2].
  [JumpAboveOrEqual] -> [^self concretizeConditionalJump: 16r3].
  [JumpAbove] -> [^self concretizeConditionalJump: 16r7].
  [JumpBelowOrEqual] -> [^self concretizeConditionalJump: 16r6].
  [JumpFPEqual] -> [^self concretizeConditionalJump: 16r4].
  [JumpFPNotEqual] -> [^self concretizeConditionalJump: 16r5].
  [JumpFPLess] -> [^self concretizeConditionalJump: 16r2].
  [JumpFPGreaterOrEqual] -> [^self concretizeConditionalJump: 16r3].
  [JumpFPGreater] -> [^self concretizeConditionalJump: 16r7].
  [JumpFPLessOrEqual] -> [^self concretizeConditionalJump: 16r6].
  [JumpFPOrdered] -> [^self concretizeConditionalJump: 16rB].
  [JumpFPUnordered] -> [^self concretizeConditionalJump: 16rA].
  [RetN] -> [^self concretizeRetN].
  [Stop] -> [^self concretizeStop].
  "Arithmetic"
  [AddCqR] -> [^self concretizeArithCqRWithRO: 0 raxOpcode: 15r05].
  [AddcCqR] -> [^self concretizeArithCqRWithRO: 2 raxOpcode: 15r15].
  [AddCwR] -> [^self concretizeArithCwR: 16r03].
  [AddRR] -> [^self concretizeOpRR: 16r03].
  [AddRsRs] -> [^self concretizeSEEOpRsRs: 16r58].
  [AddRdRd] -> [^self concretizeSEE2OpRdRd: 16r58].
  [AndCqR] -> [^self concretizeArithCqRWithRO: 4 raxOpcode: 16r25].
  [AndCwR] -> [^self concretizeArithCwR: 16r23].
  [AndRR] -> [^self concretizeOpRR: 16r23].
  [TstCqR] -> [^self concretizeTstCqR].
  [CmpCqR] -> [^self concretizeArithCqRWithRO: 7 raxOpcode: 16r3D].
  [CmpCwR] -> [^self concretizeArithCwR: 16r39].
  [CmpC32R] -> [^self concretizeCmpC32R].
  [CmpRR] -> [^self concretizeReverseOpRR: 16r39].
  [CmpRdRd] -> [^self concretizeCmpRdRd].
  [CmpRsRs] -> [^self concretizeCmpRsRs].
  [DivRdRd] -> [^self concretizeSEE2OpRdRd: 16r5E].
  [DivRsRs] -> [^self concretizeSEEOpRsRs: 16r5E].
  [MulRdRd] -> [^self concretizeSEE2OpRdRd: 16r59].
  [MulRsRs] -> [^self concretizeSEEOpRsRs: 16r59].
  [OrCqR] -> [^self concretizeArithCqRWithRO: 1 raxOpcode: 16r0D].
  [OrCwR] -> [^self concretizeArithCwR: 16r0B].
  [OrRR] -> [^self concretizeOpRR: 16r0B].
  [SubCqR] -> [^self concretizeArithCqRWithRO: 5 raxOpcode: 16r2D].
  [SubbCqR] -> [^self concretizeArithCqRWithRO: 3 raxOpcode: 16r1D].
  [SubCwR] -> [^self concretizeArithCwR: 16r2B].
  [SubRR] -> [^self concretizeOpRR: 16r2B].
  [SubRdRd] -> [^self concretizeSEE2OpRdRd: 16r5C].
  [SubRsRs] -> [^self concretizeSEEOpRsRs: 16r5C].
  [SqrtRd] -> [^self concretizeSqrtRd].
  [SqrtRs] -> [^self concretizeSqrtRs].
  [XorCwR] -> [^self concretizeArithCwR: 16r33].
  [XorRR] -> [^self concretizeOpRR: 16r33].
  [XorRdRd] -> [^self concretizeXorRdRd].
  [XorRsRs] -> [^self concretizeXorRsRs].
  [NegateR] -> [^self concretizeNegateR].
  [LoadEffectiveAddressMwrR] -> [^self concretizeLoadEffectiveAddressMwrR].
  [RotateLeftCqR] -> [^self concretizeShiftCqRegOpcode: 0].
  [RotateRightCqR] -> [^self concretizeShiftCqRegOpcode: 1].
  [ArithmeticShiftRightCqR] -> [^self concretizeShiftCqRegOpcode: 7].
  [LogicalShiftRightCqR] -> [^self concretizeShiftCqRegOpcode: 5].
  [LogicalShiftLeftCqR] -> [^self concretizeShiftCqRegOpcode: 4].
  [ArithmeticShiftRightRR] -> [^self concretizeShiftRegRegOpcode: 7].
  [LogicalShiftLeftRR] -> [^self concretizeShiftRegRegOpcode: 4].
  [ClzRR] -> [^self concretizeClzRR].
  "Data Movement"
  [MoveCqR] -> [^self concretizeMoveCqR].
  [MoveCwR] -> [^self concretizeMoveCwR].
  [MoveC32R] -> [^self concretizeMoveC32R].
  [MoveRR] -> [^self concretizeReverseOpRR: 16r89].
  [MoveAwR] -> [^self concretizeMoveAwR].
  [MoveA32R] -> [^self concretizeMoveA32R].
  [MoveRAw] -> [^self concretizeMoveRAw].
  [MoveRA32] -> [^self concretizeMoveRA32].
  [MoveAbR] -> [^self concretizeMoveAbR].
  [MoveRAb] -> [^self concretizeMoveRAb].
  [MoveMbrR] -> [^self concretizeMoveMbrR].
  [MoveRMbr] -> [^self concretizeMoveRMbr].
  [MoveM8rR] -> [^self concretizeMoveMbrR].
  [MoveRM8r] -> [^self concretizeMoveRMbr].
  [MoveM16rR] -> [^self concretizeMoveM16rR].
  [MoveRM16r] -> [^self concretizeMoveRM16r].
  [MoveM32rR] -> [^self concretizeMoveM32rR].
  [MoveM32rRs] -> [^self concretizeMoveM32rRs].
  [MoveM64rRd] -> [^self concretizeMoveM64rRd].
  [MoveMwrR] -> [^self concretizeMoveMwrR].
  [MoveXbrRR] -> [^self concretizeMoveXbrRR].
  [MoveRXbrR] -> [^self concretizeMoveRXbrR].
  [MoveXwrRR] -> [^self concretizeMoveXwrRR].
  [MoveRXwrR] -> [^self concretizeMoveRXwrR].
  [MoveX32rRR] -> [^self concretizeMoveX32rRR].
  [MoveRX32rR] -> [^self concretizeMoveRX32rR].
  [MoveRMwr] -> [^self concretizeMoveRMwr].
  [MoveRM32r] -> [^self concretizeMoveRM32r].
  [MoveRsM32r] -> [^self concretizeMoveRsM32r].
  [MoveRdM64r] -> [^self concretizeMoveRdM64r].
  [MoveRdR] -> [^self concretizeMoveRdR].
  [MoveRRd] -> [^self concretizeMoveRRd].
  [MoveRdRd] -> [^self concretizeMoveRdRd].
  [MoveRsRs] -> [^self concretizeMoveRsRs].
  [PopR] -> [^self concretizePopR].
  [PushR] -> [^self concretizePushR].
  [PushCq] -> [^self concretizePushCq].
  [PushCw] -> [^self concretizePushCw].
  [PrefetchAw] -> [^self concretizePrefetchAw].
  "Conversion"
  [ConvertRRd] -> [^self concretizeConvertRRd].
  [ConvertRdR] -> [^self concretizeConvertRdR].
  [ConvertRRs] -> [^self concretizeConvertRRs].
  [ConvertRsR] -> [^self concretizeConvertRsR].
  [ConvertRsRd] -> [^self concretizeConvertRsRd].
  [ConvertRdRs] -> [^self concretizeConvertRdRs].
 
  [SignExtend8RR] -> [^self concretizeSignExtend8RR].
  [SignExtend16RR] -> [^self concretizeSignExtend16RR].
  [SignExtend32RR] -> [^self concretizeSignExtend32RR].
 
  [ZeroExtend8RR] -> [^self concretizeZeroExtend8RR].
  [ZeroExtend16RR] -> [^self concretizeZeroExtend16RR].
  [ZeroExtend32RR] -> [^self concretizeZeroExtend32RR].
  }.
 
  ^0 "keep Slang happy"!

Item was changed:
  ----- Method: CogX64Compiler>>dispatchConcretizeProcessorSpecific (in category 'generate machine code') -----
  dispatchConcretizeProcessorSpecific
  "Attempt to generate concrete machine code for the instruction at address.
  This is part of the inner dispatch of concretizeAt: actualAddress which exists only
  to get around the number of literals limits in the SqueakV3 (blue book derived)
  bytecode set."
  opcode caseOf: {
+ [CPUID] -> [^self concretizeCPUID].
  "Specific Arithmetic"
  [CDQ] -> [^self concretizeCDQ].
  [IDIVR] -> [^self concretizeIDIVR].
  [IMULRR] -> [^self concretizeMulRR].
- [CPUID] -> [^self concretizeCPUID].
  [XCHGRR] -> [^self concretizeXCHGRR].
  "Specific Data Movement"
  [REP] -> [^self concretizeREP].
  [CLD] -> [^self concretizeCLD].
  [MOVSB] -> [^self concretizeMOVSB].
  [MOVSQ] -> [^self concretizeMOVSQ].
  [BSR] -> [^self concretizeBSR].
  "Multi-processing"
  "[CMPXCHGAwR] -> [^self concretizeCMPXCHGAwR].
  [CMPXCHGMwrR] -> [^self concretizeCMPXCHGMwrR].
  [LFENCE] -> [^self concretizeFENCE: 5]."
  [MFENCE] -> [^self concretizeFENCE: 6].
  [SFENCE] -> [^self concretizeFENCE: 7].
+ [LOCK] -> [^self concretizeLOCK].
- "[LOCK] -> [^self concretizeLOCK]."
  [MoveRAwNoVBR] -> [^self concretizeMoveRAwNoVBR].
+ [CMPXCHGRMr] -> [^self concretizeCMPXCHGRMr].
+ [SETE] -> [^self concretizeSet: 4].
- [XCHGAwR] -> [^self concretizeXCHGAwR].
  "[XCHGMwrR] -> [^self concretizeXCHGMwrR]."
  }.
  ^0!

Item was changed:
  ----- Method: CogX64Compiler>>genMarshallNArgs:arg:arg:arg:arg: (in category 'abi') -----
  genMarshallNArgs: numArgs arg: regOrConst0 arg: regOrConst1 arg: regOrConst2 arg: regOrConst3
+ "Generate the code to pass up to four arguments in a C run-time call.  Hack: each argument is either
+ a negative number, which encodes a positive constant, or a non-negative number, that of a register.
- "Generate the code to pass up to four arguments in a C run-time call.  Hack: each argument is
- either a negative number, which encodes a constant, or a non-negative number, that of a register.
 
+ Run-time calls have no more than four arguments, so chosen so that on ARM32, where in its C ABI
+ the first four integer arguments are passed in registers, all arguments can be passed in registers.
+ We defer to the back end to generate this code not so much that the back end knows whether it
+ uses the stack or registers to pass arguments (it does, but...). In fact we defer for an extremely evil
+ reason. Doing so allows the x64 (where up to 6 args are passed) to assign the register arguments
+ in an order that allows some of the argument registers to be used for specific abstract  registers,
+ specifically ReceiverResultReg and ClassReg.  This is evil, evil, evil, but also it's really nice to keep
+ using the old register assignments the original author has grown accustomed to.
- Run-time calls have no more than four arguments, so chosen so that on ARM, where in its C ABI the
- first four integer arguments are passed in registers, all arguments can be passed in registers.  We
- defer to the back end to generate this code not so much that the back end knows whether it uses
- the stack or registers to pass arguments (it does, but...). In fact we defer for an extremely evil reason.
- Doing so allows the x64 (where up to 6 args are passed) to assign the register arguments in an order
- that allows some of the argument registers to be used for specific abstract  registers, specifically
- ReceiverResultReg and ClassReg.  This is evil, evil, evil, but also it's really nice to keep using the old
- register assignments the original author has grown accustomed to.
 
  How can this possibly work?  Look at Cogit class>>runtime for a list of the run-time calls and their
  arguments, including which arguments are passed in which registers.  Look at CogX64Compiler's
  subclass implementations of initializeAbstractRegisters.  There are no calls in which ReceiverResultReg
  (RDX) and/or ClassReg (RCX) are passed along with Arg0Reg and Arg1Reg, and none in which the use of
  either ReceiverResultReg or ClassReg conflict for args 3 & 4.  So if args are assigned in order, the
  registers do not get overwritten.  Yes, this is evil, but it's so nice to continue to use RCX & RDX.
 
  Argument registers for args 0 to 3 in SysV are RDI RSI RDX RCX, and in Win64 are RCX RDX R8 R9"
  <inline: true>
  SysV ifFalse: "WIN64 ABI allways reserve shadow space on the stack for callee to save up to 4 register parameters"
  [cogit SubCq: 32 R: RSP].
  numArgs = 0 ifTrue: [^self].
+ self assert: numArgs <= 4.
+ (cogit isTrampolineArgConstant: regOrConst0)
+ ifTrue: [cogit MoveCq: (cogit trampolineArgValue: regOrConst0) R: CArg0Reg] "a.k.a. Arg0Reg"
- SysV
- ifTrue:
- [(cogit isTrampolineArgConstant: regOrConst0)
- ifTrue: [cogit MoveCq: (cogit trampolineArgValue: regOrConst0) R: RDI] "a.k.a. Arg0Reg"
- ifFalse:
- [regOrConst0 ~= RDI ifTrue:
- [cogit MoveR: regOrConst0 R: RDI]].
- numArgs = 1 ifTrue: [^self].
- (cogit isTrampolineArgConstant: regOrConst1)
- ifTrue: [cogit MoveCq: (cogit trampolineArgValue: regOrConst1) R: RSI] "a.k.a. Arg1Reg"
- ifFalse:
- [regOrConst1 ~= RSI ifTrue:
- [cogit MoveR: regOrConst1 R: RSI]].
- numArgs = 2 ifTrue: [^self].
- (cogit isTrampolineArgConstant: regOrConst2)
- ifTrue: [cogit MoveCq: (cogit trampolineArgValue: regOrConst2) R: RDX] "a.k.a. ReceiverResultReg"
- ifFalse:
- [regOrConst2 ~= RDX ifTrue:
- [cogit MoveR: regOrConst2 R: RDX]].
- numArgs = 3 ifTrue: [^self].
- (cogit isTrampolineArgConstant: regOrConst3)
- ifTrue: [cogit MoveCq: (cogit trampolineArgValue: regOrConst3) R: RCX] "a.k.a. ClassReg"
- ifFalse:
- [regOrConst3 ~= RCX ifTrue:
- [cogit MoveR: regOrConst3 R: RCX]]]
  ifFalse:
+ [regOrConst0 ~= CArg0Reg ifTrue:
+ [cogit MoveR: regOrConst0 R: CArg0Reg]].
+ numArgs = 1 ifTrue: [^self].
+ (cogit isTrampolineArgConstant: regOrConst1)
+ ifTrue: [cogit MoveCq: (cogit trampolineArgValue: regOrConst1) R: CArg1Reg] "a.k.a. Arg1Reg"
+ ifFalse:
+ [regOrConst1 ~= CArg1Reg ifTrue:
+ [cogit MoveR: regOrConst1 R: CArg1Reg]].
+ numArgs = 2 ifTrue: [^self].
+ (cogit isTrampolineArgConstant: regOrConst2)
+ ifTrue: [cogit MoveCq: (cogit trampolineArgValue: regOrConst2) R: CArg2Reg] "a.k.a. ReceiverResultReg (SysV) ClassReg (Win64)"
+ ifFalse:
+ [regOrConst2 ~= CArg2Reg ifTrue:
+ [cogit MoveR: regOrConst2 R: CArg2Reg]].
+ numArgs = 3 ifTrue: [^self].
+ (cogit isTrampolineArgConstant: regOrConst3)
+ ifTrue: [cogit MoveCq: (cogit trampolineArgValue: regOrConst3) R: CArg3Reg] "a.k.a. ClassReg (SysV) ReceiverResultReg (Win64)"
+ ifFalse:
+ [regOrConst3 ~= CArg3Reg ifTrue:
+ [cogit MoveR: regOrConst3 R: CArg3Reg]]!
- [(cogit isTrampolineArgConstant: regOrConst0)
- ifTrue: [cogit MoveCq: (cogit trampolineArgValue: regOrConst0) R: RCX] "a.k.a. Arg0Reg"
- ifFalse:
- [regOrConst0 ~= RCX ifTrue:
- [cogit MoveR: regOrConst0 R: RCX]].
- numArgs = 1 ifTrue: [^self].
- (cogit isTrampolineArgConstant: regOrConst1)
- ifTrue: [cogit MoveCq: (cogit trampolineArgValue: regOrConst1) R: RDX] "a.k.a. Arg1Reg"
- ifFalse:
- [regOrConst1 ~= RDX ifTrue:
- [cogit MoveR: regOrConst1 R: RDX]].
- numArgs = 2 ifTrue: [^self].
- (cogit isTrampolineArgConstant: regOrConst2)
- ifTrue: [cogit MoveCq: (cogit trampolineArgValue: regOrConst2) R: R8] "a.k.a. RISCTempReg in CogInLineLiteralsX64Compiler and Extra6Reg in CogOutOfLineLiteralsX64Compiler"
- ifFalse:
- [regOrConst2 ~= R8 ifTrue:
- [cogit MoveR: regOrConst2 R: R8]].
- numArgs = 3 ifTrue: [^self].
- (cogit isTrampolineArgConstant: regOrConst3)
- ifTrue: [cogit MoveCq: (cogit trampolineArgValue: regOrConst3) R: R9] "a.k.a. SendNumArgsReg"
- ifFalse:
- [regOrConst3 ~= R9 ifTrue:
- [cogit MoveR: regOrConst3 R: R9]]].
- self assert: numArgs <= 4!

Item was changed:
  ----- Method: CogX64Compiler>>generateLowLevelTryLock: (in category 'multi-threading') -----
  generateLowLevelTryLock: vmOwnerLockAddress
  "Generate a function that attempts to lock the vmOwnerLock and answers if it succeeded."
  <inline: true>
+ | vmOwnerLockAddressReg |
- | callerSavedReg |
  vmOwnerLockAddress = 0 ifTrue:
  [cogit
+ MoveCq: 1 R: ABIResultReg;
- MoveCq: 1 R: RAX;
  RetN: 0.
  ^self].
+ "RAX holds the value of lock if unlocked (zero), receives the existing value of the lock; RAX is implicit in CMPXCHG"
+ vmOwnerLockAddressReg := CArg1Reg.
- "There is no 64-bit absolute address form of XCHGQ.  Insteads, load the address into a scratch register."
- callerSavedReg := cogit availableRegisterOrNoneIn: (ABICallerSavedRegisterMask bitClear: 1 << RAX).
  cogit
+ MoveCq: 0 R: RAX;
+ MoveCq: vmOwnerLockAddress R: vmOwnerLockAddressReg;
+ gen: LOCK;
+ gen: CMPXCHGRMr operand: CArg0Reg operand: vmOwnerLockAddressReg;
+ gen: SETE operand: ABIResultReg;
- MoveCq: 1 R: RAX;
- gen: MFENCE; "make the XCHG globally consistent"
- gen: XCHGAwR operand: vmOwnerLockAddress operand: RAX operand: callerSavedReg;
- gen: SFENCE; "make the store globally visible"
- SubCq: 1 R: RAX; "Since we only ever set the lock to 1 or 0, subtracting 1 sets
-   RAX to 0 if the lock was already locked and non-zero if it wasn't."
  RetN: 0!

Item was changed:
+ ----- Method: CogX64Compiler>>numLowLevelLockOpcodes (in category 'multi-threading') -----
- ----- Method: CogX64Compiler>>numLowLevelLockOpcodes (in category 'accessing') -----
  numLowLevelLockOpcodes
+ <inline: #always>
  "ceTryLockVMOwner:
+ xorq %rax, %rax
+ movq &vmOwnerLock, %rsi
+ lock cmpxchgq %rdi, (%rsi) N.B. lock cmpxchgq are two separate opcodes
+ setz %alt
- movq $0x1, %rax
- mfence  : 0F AE F0
- movq &vmOwnerLockFromMachineCode, %rcx  N.B. movq,xchgq are one XCHGAwR instruction
- xchgq %rax, (%rcx)
- sfence
- subq $0x1, %rax
  ret
  ceUnlockVMOwner:
  xorq %rax, %rax
  movq %rax, &vmOwnerLockFromMachineCode
  sfence
  ret"
  ^10!

Item was removed:
- ----- Method: CogX64CompilerTests>>testCMPXCHGAwR (in category 'tests') -----
- testCMPXCHGAwR
- "self new testCMPXCHGAwR"
- | cmpxchgAwR |
- cmpxchgAwR := CogX64Compiler classPool at: #CMPXCHGAwR.
- self concreteCompilerClass registersWithNamesDo:
- [:reg :regname|
- #(16r555555 16rAAAAAA) do:
- [:addr| | inst len |
- inst := self gen: cmpxchgAwR operand: addr operand: reg.
- len := inst concretizeAt: 0.
- self processor
- disassembleInstructionAt: 0
- In: inst machineCode object
- into: [:str :sz| | plainJane herIntended |
- "Convert e.g. '00000000: movl %eax, 0x2(%eax) : 89 40 02' to  'movl %eax, 0x2(%eax)'"
- plainJane := self strip: str.
- herIntended := 'cmpxchgl ', regname, ', 0x', (addr hex allButFirst: 3).
- self assert: herIntended equals: plainJane.
- self assert: len = sz]]]!

Item was removed:
- ----- Method: CogX64CompilerTests>>testXCHGAwR (in category 'tests') -----
- testXCHGAwR
- "self new testXCHGAwR"
- | xchgAwR |
- xchgAwR := CogX64Compiler classPool at: #XCHGAwR.
- self concreteCompilerClass registersWithNamesDo:
- [:reg :regname|
- #(16r555555 16rAAAAAA) do:
- [:addr| | inst len |
- inst := self gen: xchgAwR operand: addr operand: reg.
- len := inst concretizeAt: 0.
- self processor
- disassembleInstructionAt: 0
- In: inst machineCode object
- into: [:str :sz| | plainJane herIntended |
- "Convert e.g. '00000000: movl %eax, 0x2(%eax) : 89 40 02' to  'movl %eax, 0x2(%eax)'"
- plainJane := self strip: str.
- herIntended := 'xchgl ', regname, ', 0x', (addr hex allButFirst: 3).
- self assert: herIntended equals: plainJane.
- self assert: len = sz]]]!

Item was removed:
- ----- Method: CogX64CompilerTests>>testXCHGMwrR (in category 'tests') -----
- testXCHGMwrR
- "self new testXCHGMwrR"
- | xchgMwrR |
- xchgMwrR := CogX64Compiler classPool at: #XCHGMwrR.
- self concreteCompilerClass registersWithNamesDo:
- [:sreg :srname|
- self concreteCompilerClass registersWithNamesDo:
- [:dreg :drname|
- ((1 to: 19 by: 3) collect: [:po2| 2 raisedToInteger: po2]) do:
- [:offset| | memory |
- self resetGen.
- self gen: xchgMwrR operand: offset operand: sreg operand: dreg.
- memory := self generateInstructions.
- self processor
- disassembleInstructionAt: 0
- In: memory
- into: [:str :sz| | plainJane herIntended |
- plainJane := self strip: str.
- herIntended := 'xchgl ', drname, ', 0x', (offset hex allButFirst: 3), '(', srname, ')'.
- self assert: herIntended equals: plainJane.
- self assert: memory size = sz]]]]!

Item was removed:
- ----- Method: Cogit>>tryLockVMOwner (in category 'multi-threading') -----
- tryLockVMOwner
- <api>
- "ceTryLockVMOwner does an atomic swap of the lock with 1 and
- then subtracts 1from lock's value.  So if the result is 0 the lock was
- already held.  Anything else (in fact -1) implies we hold the lock."
- <cmacro: '() (ceTryLockVMOwner() !!= 0)'>
- ^(self simulateLeafCallOf: ceTryLockVMOwner) ~= 0!

Item was added:
+ ----- Method: Cogit>>tryLockVMOwner: (in category 'multi-threading') -----
+ tryLockVMOwner: value
+ <api>
+ "ceTryLockVMOwner does an atomic compare-and-swap of the lock
+ with the argument and zero, setting the lock to value if it was zero.
+ It answers non-zero if the lock was zero."
+ <cmacro: '(value) ceTryLockVMOwner(value)'>
+ ^(self simulateLeafCallOf: ceTryLockVMOwner) ~= 0!

Item was changed:
  ----- Method: VMStructType class>>printTypedefOn: (in category 'translation') -----
  printTypedefOn: aStream
  aStream nextPutAll: 'typedef struct '.
  self needsTypeTag ifTrue:
  [aStream nextPutAll: self structTagName; space].
  aStream nextPut: ${; cr.
  self instVarNamesAndTypesForTranslationDo:
  [:ivn :typeArg| | type |
  ivn first == $#
  ifTrue: [aStream nextPutAll: ivn]
  ifFalse:
  [type := typeArg.
  #(BytesPerWord BaseHeaderSize BytesPerOop) do:
  [:sizeConstant| | index sizeConstantSize |
  (type notNil
  and: [(index := type indexOf: sizeConstant ifAbsent: 0) > 0]) ifTrue:
  [sizeConstantSize  := VMBasicConstants classPool at: sizeConstant.
  type := (type at: index + 1) = sizeConstantSize ifTrue:
  [type := type copyReplaceFrom: index to: index + 1 with: #().
  type size = 1 ifTrue: [type first] ifFalse: [type]]]].
  type ifNotNil:
  [type isArray
  ifTrue:
  [aStream tab: 1.
  aStream nextPutAll: type first.
  (type first last isSeparator or: [type first last = $*]) ifFalse:
  [aStream tab: 2].
  aStream nextPutAll: ivn.
+ type last first isAlphaNumeric ifTrue:
- type last first isSeparator ifFalse:
  [aStream space].
  aStream nextPutAll: type last]
  ifFalse:
  [aStream tab: 1.
  aStream nextPutAll: type.
+ type last isAlphaNumeric ifTrue:
- (type last isSeparator or: [type last = $*]) ifFalse:
  [aStream tab: 1].
  aStream nextPutAll: ivn]].
  aStream nextPut: $;].
  aStream cr].
  aStream
  nextPutAll: ' } ';
  nextPutAll: self structTypeName;
  nextPut: $;;
  cr.
  self name ~= self structTypeName ifTrue:
  [(self withAllSuperclasses copyUpThrough: (self class whichClassIncludesSelector: #structTypeName) theNonMetaClass) do:
  [:structClass|
  aStream cr; nextPutAll: '#define '; nextPutAll: structClass name; space; nextPutAll: self structTypeName].
  aStream cr].
  aStream flush!