VM Maker: VMMaker.oscog-eem.2167.mcz

 
Eliot Miranda uploaded a new version of VMMaker to project VM Maker:
http://source.squeak.org/VMMaker/VMMaker.oscog-eem.2167.mcz

==================== Summary ====================

Name: VMMaker.oscog-eem.2167
Author: eem
Time: 19 March 2017, 1:50:35.950224 pm
UUID: eff5a996-2b02-4ebf-85ff-fd672ba46fc4
Ancestors: VMMaker.oscog-eem.2166

StackInterpreter:
Add the statIdleUsecs parameter to track time in the internals of ioRelinquishProcessorForMicroseconds and hence allow the system reporter to render the GC times meaningful.  Platform code (to be committed soon) invokes addIdleUsecs to tally idle time.

=============== Diff against VMMaker.oscog-eem.2166 ===============

Item was changed:
  InterpreterPrimitives subclass: #StackInterpreter
+ instanceVariableNames: 'currentBytecode bytecodeSetSelector localFP localIP localSP stackLimit stackPage stackPages method instructionPointer stackPointer framePointer localReturnValue localAbsentReceiver localAbsentReceiverOrZero extA extB numExtB primitiveFunctionPointer methodCache nsMethodCache atCache lkupClassTag lkupClass methodDictLinearSearchLimit highestRunnableProcessPriority nextWakeupUsecs nextPollUsecs inIOProcessEvents interruptKeycode interruptPending savedWindowSize imageHeaderFlags fullScreenFlag deferDisplayUpdates pendingFinalizationSignals extraVMMemory interpreterProxy showSurfaceFn primitiveTable primitiveAccessorDepthTable externalPrimitiveTable externalPrimitiveTableFirstFreeIndex overflowedPage extraFramesToMoveOnOverflow globalSessionID jmpBuf jmpDepth suspendedCallbacks suspendedMethods numStackPages desiredNumStackPages desiredEdenBytes classNameIndex thisClassIndex metaclassNumSlots interruptCheckChain suppressHeartbeatFlag breakSelector breakSelector
 Length longRunningPrimitiveCheckMethod longRunningPrimitiveCheckSemaphore longRunningPrimitiveStartUsecs longRunningPrimitiveStopUsecs longRunningPrimitiveGCUsecs longRunningPrimitiveCheckSequenceNumber longRunningPrimitiveSignalUndelivered checkAllocFiller tempOop tempOop2 tempOop3 theUnknownShort the2ndUnknownShort imageFloatsBigEndian maxExtSemTabSizeSet lastMethodCacheProbeWrite gcSemaphoreIndex classByteArrayCompactIndex checkedPluginName nativeSP nativeStackPointer lowcodeCalloutState shadowCallStackPointer statForceInterruptCheck statStackOverflow statStackPageDivorce statCheckForEvents statProcessSwitch statIOProcessEvents statPendingFinalizationSignals statIdleUsecs debugCallbackPath debugCallbackReturns debugCallbackInvokes'
- instanceVariableNames: 'currentBytecode bytecodeSetSelector localFP localIP localSP stackLimit stackPage stackPages method instructionPointer stackPointer framePointer localReturnValue localAbsentReceiver localAbsentReceiverOrZero extA extB numExtB primitiveFunctionPointer methodCache nsMethodCache atCache lkupClassTag lkupClass methodDictLinearSearchLimit highestRunnableProcessPriority nextWakeupUsecs nextPollUsecs inIOProcessEvents interruptKeycode interruptPending savedWindowSize imageHeaderFlags fullScreenFlag deferDisplayUpdates pendingFinalizationSignals extraVMMemory interpreterProxy showSurfaceFn primitiveTable primitiveAccessorDepthTable externalPrimitiveTable externalPrimitiveTableFirstFreeIndex overflowedPage extraFramesToMoveOnOverflow globalSessionID jmpBuf jmpDepth suspendedCallbacks suspendedMethods numStackPages desiredNumStackPages desiredEdenBytes classNameIndex thisClassIndex metaclassNumSlots interruptCheckChain suppressHeartbeatFlag breakSelector breakSelector
 Length longRunningPrimitiveCheckMethod longRunningPrimitiveCheckSemaphore longRunningPrimitiveStartUsecs longRunningPrimitiveStopUsecs longRunningPrimitiveGCUsecs longRunningPrimitiveCheckSequenceNumber longRunningPrimitiveSignalUndelivered checkAllocFiller tempOop tempOop2 tempOop3 theUnknownShort the2ndUnknownShort imageFloatsBigEndian maxExtSemTabSizeSet lastMethodCacheProbeWrite gcSemaphoreIndex classByteArrayCompactIndex checkedPluginName statForceInterruptCheck statStackOverflow statStackPageDivorce statCheckForEvents statProcessSwitch statIOProcessEvents statPendingFinalizationSignals nativeSP nativeStackPointer lowcodeCalloutState shadowCallStackPointer debugCallbackPath debugCallbackReturns debugCallbackInvokes'
  classVariableNames: 'AccessModifierPrivate AccessModifierProtected AccessModifierPublic AltBytecodeEncoderClassName AltLongStoreBytecode AlternateHeaderHasPrimFlag AlternateHeaderIsOptimizedFlag AlternateHeaderNumLiteralsMask AtCacheFixedFields AtCacheFmt AtCacheMask AtCacheOop AtCacheSize AtCacheTotalSize AtPutBase BytecodeEncoderClassName BytecodeTable CacheProbeMax DirBadPath DirEntryFound DirNoMoreEntries DumpStackOnLowSpace EnclosingMixinIndex EnclosingObjectIndex EnforceAccessControl FailImbalancedPrimitives LongStoreBytecode MaxExternalPrimitiveTableSize MaxJumpBuf MaxPrimitiveIndex MaxQuickPrimitiveIndex MethodHeaderArgCountShift MethodHeaderFlagBitPosition MethodHeaderTempCountShift MixinIndex PrimNumberDoExternalCall PrimNumberExternalCall PrimNumberFFICall PrimitiveExternalCallIndex PrimitiveTable StackPageReachedButUntraced StackPageTraceInvalid StackPageTraced StackPageUnreached V3PrimitiveBitsMask'
  poolDictionaries: 'VMBasicConstants VMBytecodeConstants VMMethodCacheConstants VMObjectIndices VMSpurObjectRepresentationConstants VMSqueakClassIndices VMStackFrameOffsets'
  category: 'VMMaker-Interpreter'!
 
  !StackInterpreter commentStamp: 'eem 12/5/2014 11:32' prior: 0!
  This class is a complete implementation of the Smalltalk-80 virtual machine, derived originally from the Blue Book specification but quite different in some areas.  This VM supports Closures but *not* old-style BlockContexts.
 
  It has been modernized with 32-bit pointers, better management of Contexts (see next item), and attention to variable use that allows the CCodeGenerator (qv) to keep, eg, the instruction pointer and stack pointer in registers as well as keeping most simple variables in a global array that seems to improve performance for most platforms.
 
  The VM does not use Contexts directly.  Instead Contexts serve as proxies for a more conventional stack format that is invisible to the image.  There is considerable explanation at http://www.mirandabanda.org/cogblog/2009/01/14/under-cover-contexts-and-the-big-frame-up.  The VM maintains a fixed-size stack zone divided into pages, each page being capable of holding several method/block activations.  A send establishes a new frame in the current stack page, a return returns to the previous frame.  This eliminates allocation/deallocation of contexts and the moving of receiver and arguments from caller to callee on each send/return.  Contexts are created lazily when an activation needs a context (creating a block, explicit use of thisContext, access to sender when sender is a frame, or linking of stack pages together).  Contexts are either conventional and heap-resident ("single") or "married" and serve as proxies for their corresponding frame or "widowed", meaning that their spouse f
 rame has been returned from (died).  A married context is specially marked (more details in the code) and refers to its frame.  Likewise a married frame is specially marked and refers to its context.
 
  In addition to SmallInteger arithmetic and Floats, the VM supports logic on 32-bit PositiveLargeIntegers, thus allowing it to simulate itself much more effectively than would otherwise be the case.
 
  StackInterpreter and subclasses support multiple memory managers.  Currently there are two.  NewMemoryManager is a slightly refined version of ObjectMemory, and is the memory manager and garbage collector for the original Squeak object representation as described in "Back to the Future The Story of Squeak, A Practical Smalltalk Written in Itself", see http://ftp.squeak.org/docs/OOPSLA.Squeak.html.  Spur is a faster, more regular object representation that is designed for more performance and functionality, and to have a common header format for both 32-bit and 64-bit versions.  You can read about it in SpurMemoryManager's class comment.  There is also a video of a presentation at ESUG 2014 (https://www.youtube.com/watch?v=k0nBNS1aHZ4), along with slides (http://www.slideshare.net/esug/spur-a-new-object-representation-for-cog?related=1).!

Item was changed:
  ----- Method: StackInterpreter class>>declareCVarsIn: (in category 'translation') -----
  declareCVarsIn: aCCodeGenerator
  | vmClass |
  self class == thisContext methodClass ifFalse: [^self]. "Don't duplicate decls in subclasses"
  vmClass := aCCodeGenerator vmClass. "Generate primitiveTable etc based on vmClass, not just StackInterpreter"
  aCCodeGenerator
  addHeaderFile:'<stddef.h> /* for e.g. alloca */';
  addHeaderFile:'<setjmp.h>';
  addHeaderFile:'<wchar.h> /* for wint_t */';
  addHeaderFile:'"vmCallback.h"';
  addHeaderFile:'"sqMemoryFence.h"';
  addHeaderFile:'"dispdbg.h"'.
  LowcodeVM ifTrue: [ aCCodeGenerator addHeaderFile:'"sqLowcodeFFI.h"'].
 
  vmClass declareInterpreterVersionIn: aCCodeGenerator defaultName: 'Stack'.
  aCCodeGenerator
  var: #interpreterProxy  type: #'struct VirtualMachine*'.
  aCCodeGenerator
  declareVar: #sendTrace type: 'volatile int';
  declareVar: #byteCount type: #usqInt.
  "These need to be pointers or unsigned."
  self declareC: #(instructionPointer method newMethod)
  as: #usqInt
  in: aCCodeGenerator.
  "These are all pointers; char * because Slang has no support for C pointer arithmetic."
  self declareC: #(localIP localSP localFP nativeSP stackPointer framePointer stackLimit breakSelector nativeStackPointer nativeFramePointer shadowCallStack)
  as: #'char *'
  in: aCCodeGenerator.
  aCCodeGenerator
  var: #breakSelectorLength
  declareC: 'sqInt breakSelectorLength = MinSmallInteger'.
  self declareC: #(stackPage overflowedPage)
  as: #'StackPage *'
  in: aCCodeGenerator.
  aCCodeGenerator removeVariable: 'stackPages'.  "this is an implicit receiver in the translated code."
  "This defines bytecodeSetSelector as 0 if MULTIPLEBYTECODESETS
  is not defined, for the benefit of the interpreter on slow machines."
  aCCodeGenerator addConstantForBinding: (self bindingOf: #MULTIPLEBYTECODESETS).
  MULTIPLEBYTECODESETS == false ifTrue:
  [aCCodeGenerator
  removeVariable: 'bytecodeSetSelector'].
  BytecodeSetHasExtensions == false ifTrue:
  [aCCodeGenerator
  removeVariable: 'extA';
  removeVariable: 'extB'].
  aCCodeGenerator
  var: #methodCache
  declareC: 'sqIntptr_t methodCache[MethodCacheSize + 1 /* ', (MethodCacheSize + 1) printString, ' */]'.
  NewspeakVM
  ifTrue:
  [aCCodeGenerator
  var: #nsMethodCache
  declareC: 'sqIntptr_t nsMethodCache[NSMethodCacheSize + 1 /* ', (NSMethodCacheSize + 1) printString, ' */]']
  ifFalse:
  [aCCodeGenerator
  removeVariable: 'localAbsentReceiver';
  removeVariable: 'localAbsentReceiverOrZero'].
  AtCacheTotalSize isInteger ifTrue:
  [aCCodeGenerator
  var: #atCache
  declareC: 'sqInt atCache[AtCacheTotalSize + 1 /* ', (AtCacheTotalSize + 1) printString, ' */]'].
  aCCodeGenerator
  var: #primitiveTable
  declareC: 'void (*primitiveTable[MaxPrimitiveIndex + 2 /* ', (MaxPrimitiveIndex + 2) printString, ' */])(void) = ', vmClass primitiveTableString.
  vmClass primitiveTable do:
  [:symbolOrNot|
  (symbolOrNot isSymbol
  and: [symbolOrNot ~~ #primitiveFail]) ifTrue:
  [(aCCodeGenerator methodNamed: symbolOrNot) ifNotNil:
  [:tMethod| tMethod returnType: #void]]].
  vmClass objectMemoryClass hasSpurMemoryManagerAPI
  ifTrue:
  [aCCodeGenerator
  var: #primitiveAccessorDepthTable
  type: 'signed char'
  sizeString: 'MaxPrimitiveIndex + 2 /* ', (MaxPrimitiveIndex + 2) printString, ' */'
  array: vmClass primitiveAccessorDepthTable]
  ifFalse:
  [aCCodeGenerator removeVariable: #primitiveAccessorDepthTable].
  aCCodeGenerator
  var: #primitiveFunctionPointer
  declareC: 'void (*primitiveFunctionPointer)()'.
  aCCodeGenerator
  var: #externalPrimitiveTable
  declareC: 'void (*externalPrimitiveTable[MaxExternalPrimitiveTableSize + 1 /* ', (MaxExternalPrimitiveTableSize + 1) printString, ' */])(void)'.
  aCCodeGenerator var: #showSurfaceFn type: #'void *'.
  aCCodeGenerator
  var: #jmpBuf
  declareC: 'jmp_buf jmpBuf[MaxJumpBuf + 1 /* ', (MaxJumpBuf + 1) printString, ' */]'.
  aCCodeGenerator
  var: #suspendedCallbacks
  declareC: 'usqInt suspendedCallbacks[MaxJumpBuf + 1 /* ', (MaxJumpBuf + 1) printString, ' */]'.
  aCCodeGenerator
  var: #suspendedMethods
  declareC: 'usqInt suspendedMethods[MaxJumpBuf + 1 /* ', (MaxJumpBuf + 1) printString, ' */]'.
  aCCodeGenerator
  var: #interruptCheckChain
  declareC: 'void (*interruptCheckChain)(void) = 0'.
 
  self declareCAsUSqLong: #(nextPollUsecs nextWakeupUsecs longRunningPrimitiveGCUsecs
  longRunningPrimitiveStartUsecs longRunningPrimitiveStopUsecs
  "these are high-frequency enough that they're overflowing quite quickly on modern hardware"
  statProcessSwitch statIOProcessEvents statForceInterruptCheck
+ statCheckForEvents statStackOverflow statStackPageDivorce
+ statIdleUsecs)
- statCheckForEvents statStackOverflow statStackPageDivorce)
  in: aCCodeGenerator.
  aCCodeGenerator var: #nextProfileTick type: #sqLong.
 
+ LowcodeVM ifTrue:
+ [aCCodeGenerator
- LowcodeVM ifTrue: [
- aCCodeGenerator
  var: #shadowCallStackPointer
+ type: #'char *'.
- type: #'char*'.
  aCCodeGenerator
  var: #lowcodeCalloutState
+ type: #'sqLowcodeCalloutState*']!
- type: #'sqLowcodeCalloutState*'
- ].!

Item was added:
+ ----- Method: StackInterpreter>>addIdleUsecs: (in category 'primitive support') -----
+ addIdleUsecs: idleUsecs
+ "The various poll/select calls in the VM should attempt to tally the ammount
+ of time spent at idle here, so as to render the uptime value meaningful."
+ <export: true>
+ statIdleUsecs := statIdleUsecs + idleUsecs!

Item was changed:
  ----- Method: StackInterpreter>>initialize (in category 'initialization') -----
  initialize
  "Here we can initialize the variables C initializes to zero.  #initialize methods do /not/ get translated."
  checkAllocFiller := false. "must precede initializeObjectMemory:"
  primFailCode := 0.
  stackLimit := 0. "This is also the initialization flag for the stack system."
  stackPage := overflowedPage := 0.
  extraFramesToMoveOnOverflow := 0.
  bytecodeSetSelector := 0.
  highestRunnableProcessPriority := 0.
  nextProfileTick := 0.
  nextPollUsecs := 0.
  nextWakeupUsecs := 0.
  tempOop := tempOop2 := tempOop3 := theUnknownShort := 0.
  interruptPending := false.
  inIOProcessEvents := 0.
  fullScreenFlag := 0.
  deferDisplayUpdates := false.
  pendingFinalizationSignals := statPendingFinalizationSignals := 0.
  globalSessionID := 0.
  jmpDepth := 0.
  longRunningPrimitiveStartUsecs := longRunningPrimitiveStopUsecs := 0.
  maxExtSemTabSizeSet := false.
  debugCallbackInvokes := debugCallbackPath := debugCallbackReturns := 0.
  statForceInterruptCheck := statStackOverflow := statCheckForEvents :=
+ statProcessSwitch := statIOProcessEvents := statStackPageDivorce :=
+ statIdleUsecs := 0!
- statProcessSwitch := statIOProcessEvents := statStackPageDivorce := 0!

Item was changed:
  ----- Method: StackInterpreterPrimitives>>primitiveVMParameter (in category 'system control primitives') -----
(excessive size, no diff calculated)