The Inbox: Compiler-tobe.455.mcz

A new version of Compiler was added to project The Inbox:
http://source.squeak.org/inbox/Compiler-tobe.455.mcz

==================== Summary ====================

Name: Compiler-tobe.455
Author: tobe
Time: 11 March 2021, 1:50:12.615385 pm
UUID: ba3e2f61-8c2a-4dcf-a288-f6e81db4eda1
Ancestors: Compiler-codefrau.454

Fix scanAllTokenPositionsInto: when the text of a comment happened to appear just before it. See testScanTokensRepeatComment for an example.

=============== Diff against Compiler-codefrau.454 ===============

Item was changed:
  ----- Method: Scanner>>scanAllTokenPositionsInto: (in category 'expression types') -----
  scanAllTokenPositionsInto: aBlock
         "Evaluate aBlock with the start and end positions of all separate non-white-space tokens, including comments."
 
         | lastMark |
         lastMark := 1.
         [currentComment ifNotNil:
                 [currentComment do:
                         [:cmnt| | idx |
+                         idx := source originalContents findLastOccurrenceOfString: cmnt startingAt: lastMark.
-                         idx := source originalContents indexOfSubCollection: cmnt startingAt: lastMark.
                          (idx > 0 and: [idx < mark]) ifTrue:
                                 [aBlock value: idx - 1 value: (lastMark := idx + cmnt size)]].
                  currentComment := nil].
         mark ifNotNil:
                 [(token == #-
                   and: [(self typeTableAt: hereChar) == #xDigit]) ifTrue:
                         [| savedMark |
                          savedMark := mark.
                          self scanToken.
                          token := token negated.
                          mark := savedMark].
                 "Compensate for the fact that the parser uses two character lookahead.  Normally we must
                   remove the extra two characters.  But this mustn't happen for the last token at the end of stream."
                  aBlock
                         value: mark
                         value: source position - (aheadChar == DoItCharacter
                                 ifTrue: [hereChar == DoItCharacter
                                         ifTrue: [0]
                                         ifFalse: [1]]
                                 ifFalse: [2])].
          (tokenType == #rightParenthesis
           or: [tokenType == #doIt]) ifTrue:
                 [^self].
         tokenType == #leftParenthesis
                 ifTrue:
                         [self scanToken; scanAllTokenPositionsInto: aBlock]
                 ifFalse:
                         [(tokenType == #word or: [tokenType == #keyword or: [tokenType == #colon]])
                                 ifTrue:
                                         [self scanLitWord.
                                          token == #true ifTrue: [token := true].
                                          token == #false ifTrue: [token := false].
                                          token == #nil ifTrue: [token := nil]]
                                 ifFalse:
                                         [(token == #-
                                           and: [(self typeTableAt: hereChar) == #xDigit])
                                                 ifTrue:
                                                         [self scanToken.
                                                          token := token negated]]].
                 self scanToken ] repeat!