The Inbox: ShoutCore-ul.80.mcz

Levente Uzonyi uploaded a new version of ShoutCore to project The Inbox:
http://source.squeak.org/inbox/ShoutCore-ul.80.mcz

==================== Summary ====================

Name: ShoutCore-ul.80
Author: ul
Time: 14 June 2020, 12:12:00.544627 am
UUID: e6ef45f1-c973-4d64-9d91-8c4a80257173
Ancestors: ShoutCore-mt.79

SHParserST80 performance tweaks:
- cache size of source in a new instance variable: sourceSize
- #currentChar, #nextChar and #peekChar send #at: instead of #at:ifAbsent: to source.
- use sourceSize instead of source size

=============== Diff against ShoutCore-mt.79 ===============

Item was changed:
  Object subclass: #SHParserST80
+ instanceVariableNames: 'classOrMetaClass source sourceSize workspace arguments sourcePosition currentToken currentTokenFirst temporaries instanceVariables errorBlock currentTokenSourcePosition bracketDepth ranges environment allowUnderscoreAssignments allowUnderscoreSelectors allowBlockArgumentAssignment parseAMethod currentTokenType context'
- instanceVariableNames: 'classOrMetaClass source workspace arguments sourcePosition currentToken currentTokenFirst temporaries instanceVariables errorBlock currentTokenSourcePosition bracketDepth ranges environment allowUnderscoreAssignments allowUnderscoreSelectors allowBlockArgumentAssignment parseAMethod currentTokenType context'
  classVariableNames: ''
  poolDictionaries: ''
  category: 'ShoutCore-Parsing'!
 
+ !SHParserST80 commentStamp: 'ul 6/14/2020 00:05' prior: 0!
- !SHParserST80 commentStamp: 'ul 7/30/2019 00:31' prior: 0!
  I am a Smalltalk method / expression parser.
 
  Rather than creating an Abstract Syntax Tree, I create a sequence of SHRanges (in my 'ranges' instance variable), which represent the tokens within the String I am parsing.
 
  I am used by a SHTextStylerST80 to parse method source strings.
  I am able to parse incomplete / incorrect methods, and so can be used to parse methods that are being edited.
 
  Instance Variables
  allowBlockArgumentAssignment: <Boolean>
  allowUnderscoreAssignments: <Boolean>
  allowUnderscoreSelectors: <Boolean>
  arguments: <OrderedCollection<OrderedCollection<String>|nil>
  bracketDepth: <Integer>
  classOrMetaClass: <Class|nil>
  currentToken: <String|nil>
  currentTokenFirst: <Character>
  currentTokenSourcePosition: <Integer|nil>
  currentTokenType: <Symbol|nil>
  environment: <Environment>
  errorBlock: <Block>
  instanceVariables: <Array>
  parseAMethod: <Boolean>
  ranges: <OrderedCollection<SHRange>>
  source: <String>
  sourcePosition: <Integer>
+ sourceSIze: <Integer>
  temporaries: <OrderedCollection<OrderedCollection<String>|nil>
  workspace: <Workspace|nil>
  context: <Context|nil>
 
  allowBlockArgumentAssignment
  The value cached at the beginning of parsing of Scanner allowBlockArgumentAssignment.
 
  allowUnderscoreAssignments
  The value cached at the beginning of parsing of Scanner allowUnderscoreAsAssignment.
 
  allowUnderscoreSelectors
  The value cached at the beginning of parsing of Scanner prefAllowUnderscoreSelectors.
 
  arguments
  This OrderedCollection has an element for each scope encapsulating the current scope.
  The current scope's arguments are stored in the last element. The first element holds the outermost scope's arguments.
  Each element is nil when the corresponding scope doesn't have any arguments, and the element is an OrderedCollection with the names of the arguments declared at the given scope when there's at least one.
  The size of this variable is the same as the size of temporaries.
 
  bracketDepth
  Stores the number of unclosed brackets "("  and parentheses "[" before the current sourcePosition.
 
  classOrMetaClass
  The Class or MetaClass instance, class and pool variables should be looked up during parsing or nil when not parsing code in the context of a class (e.g. when parsing code written in a Workspace). Having this set doesn't mean a method is being parsed.
 
  currentToken
  The token being analyzed for which the next range should be created for.
 
  currentTokenFirst
  The first character of currentToken cached for quick access or a space character when there are no more tokens to parse.
  Being always a Character helps avoiding extra checks.
 
  currentTokenSourcePosition
  The position of source the current token starts at or nil when there are no more tokens to process.
 
  currentTokenType
  The type of the current token calculated lazily by #currentTokenType. When it has been calculated, Its value is one of #keyword, #assignment, #ansiAssignment, #binary, #name, #other and occasionally #invalid.
 
  environment
  The Environment globals and classes should be looked up at during parsing when classOrMetaClass is nil. Its value is Smalltalk globals by default.
 
  errorBlock
  A block used to quickly stop parsing in case of an unrecoverable parse error.
 
  instanceVariables
  An Array with the instance variable names of classOrMetaClass or an empty Array when classOrMetaClass is nil.
 
  parseAMethod
  A way to tell the parser to parse source as a code snippet instead of a method. Mainly used by inspectors.
 
  ranges
  The SHRanges parsed by the parser.
 
  source
  The source code as a String to be parsed.
 
  sourcePosition
  souce is treated as a stream by the parser. This variable stores the stream position.
+
+ sourceSize
+ The size of source cached when source is assigned.
 
  temporaries
  This OrderedCollection has an element for each scope encapsulating the current scope.
  The current scope's temporaries are stored in the last element. The first element holds the outermost scope's temporaries.
  Each element is nil when the corresponding scope doesn't have any temporary variables, and the element is an OrderedCollection with the names of the temporaries declared at the given scope when there's at least one.
  The size of this variable is the same as the size of arguments.
 
  workspace
  The Workspace in whose context variables should be looked up during parsing or nil when not parsing code in a workspace.
 
  context
  The Context in which variables should be looked up during parsing or nil when not parsing within a context.
 
  Example (explore it):
 
  ranges := SHParserST80 new
  classOrMetaClass: Object;
  source: 'testMethod ^self';
  parse;
  ranges
 
  Benchmark (print it):
 
  SHParserST80 benchmark!

Item was changed:
  ----- Method: SHParserST80>>currentChar (in category 'scan') -----
  currentChar
+
+ ^sourcePosition <= sourceSize ifTrue: [ source at: sourcePosition ]!
- ^source at: sourcePosition ifAbsent: nil!

Item was changed:
  ----- Method: SHParserST80>>fail (in category 'error handling') -----
  fail
 
  | start |
  start := (ranges isEmpty ifTrue: [ 1 ] ifFalse: [ ranges last end + 1 ]).
+ start <= sourceSize ifTrue: [
- start <= source size ifTrue: [
  self
  addRangeType: #excessCode
  start: start
+ end: sourceSize ].
- end: source size ].
  errorBlock value!

Item was changed:
  ----- Method: SHParserST80>>initializeVariablesFromContext (in category 'parse support') -----
  initializeVariablesFromContext
 
  | contextSourcePcIndex contextSourceParser |
  contextSourcePcIndex := (context debuggerMap
  rangeForPC: (context isDead ifTrue: [context endPC] ifFalse: [context pc])
  in: context method
  contextIsActiveContext: true "... to really use the context's pc.")
  start.
  contextSourceParser := self class new
  classOrMetaClass: context method methodClass;
  environment: self environment;
+ source: (context method getSource asString first: contextSourcePcIndex);
- source: (context method getSource first: contextSourcePcIndex);
  yourself.
  contextSourceParser parse.
  arguments := contextSourceParser activeArguments.
  temporaries  := contextSourceParser activeTemporaries.!

Item was changed:
  ----- Method: SHParserST80>>nextChar (in category 'scan') -----
  nextChar
 
+ ^(sourcePosition := sourcePosition + 1) <= sourceSize
+ ifTrue: [ source at: sourcePosition ]
+ ifFalse: [ $  ]!
- ^source at: (sourcePosition := sourcePosition + 1) ifAbsent: $ !

Item was changed:
  ----- Method: SHParserST80>>parseString (in category 'parse') -----
  parseString
 
  | stringStart |
  stringStart := sourcePosition - 1.
  [
  (sourcePosition := source indexOf: $' startingAt: sourcePosition) = 0 ifTrue: [
  self
+ addRangeType: #unfinishedString start: stringStart end: sourceSize;
- addRangeType: #unfinishedString start: stringStart end: source size;
  fail ": 'unfinished string'"].
  self peekChar == $'
  ifTrue: [
  sourcePosition := sourcePosition + 1.
  true ]
  ifFalse: [ false ] ]
  whileTrue: [ sourcePosition := sourcePosition + 1 ].
  sourcePosition := sourcePosition + 1.
  self scanPast: #string start: stringStart end: sourcePosition - 1!

Item was changed:
  ----- Method: SHParserST80>>parseSymbolString (in category 'parse') -----
  parseSymbolString
  | first c last |
  first := sourcePosition.
  self nextChar.
  [(c := self currentChar)
  ifNil: [
+ self addRangeType: #unfinishedString start: first end: sourceSize.
- self addRangeType: #unfinishedString start: first end: source size.
  self fail ": 'unfinished string'"].
  c ~~ $' or: [
  self peekChar == $'
  ifTrue: [sourcePosition := sourcePosition + 1.true]
  ifFalse: [false]]
  ] whileTrue: [sourcePosition := sourcePosition + 1].
  last := sourcePosition.
  self
  nextChar;
  scanPast: #stringSymbol start: first - 1 end: last!

Item was changed:
  ----- Method: SHParserST80>>peekChar (in category 'scan') -----
  peekChar
 
+ ^sourcePosition < sourceSize
+ ifTrue: [ source at: sourcePosition + 1 ]
+ ifFalse: [ $  ]!
- ^source at: sourcePosition + 1 ifAbsent: $ !

Item was changed:
  ----- Method: SHParserST80>>scanComment (in category 'scan') -----
  scanComment
 
  | start |
  start := sourcePosition.
  (sourcePosition := source indexOf: $" startingAt: start + 1) = 0 ifTrue: [
+ sourcePosition := sourceSize + 1.
- sourcePosition := source size + 1.
  ^self
+ addRangeType: #unfinishedComment start: start end: sourceSize;
- addRangeType: #unfinishedComment start: start end: source size;
  fail ].
  self
  addRangeType: #comment start: start end: sourcePosition;
  nextChar;
  scanWhitespace!

Item was changed:
  ----- Method: SHParserST80>>scanWhitespace (in category 'scan') -----
  scanWhitespace
 
  (self currentChar ifNil: [ ^self ]) isSeparator ifTrue: [
  sourcePosition := source
  indexOfAnyOf: CharacterSet nonSeparators
  startingAt: sourcePosition + 1.
  sourcePosition = 0 ifTrue: [ "Not found"
+ sourcePosition := sourceSize + 1 ] ].
- sourcePosition := source size + 1 ] ].
  self currentChar == $" ifTrue: [ self scanComment ]!

Item was changed:
  ----- Method: SHParserST80>>source: (in category 'accessing') -----
  source: aStringOrText
 
+ source := aStringOrText asString.
+ sourceSize := source size!
- source := aStringOrText asString!