The Trunk: Compiler-nice.94.mcz

Nicolas Cellier uploaded a new version of Compiler to project The Trunk:
http://source.squeak.org/trunk/Compiler-nice.94.mcz

==================== Summary ====================

Name: Compiler-nice.94
Author: nice
Time: 27 October 2009, 12:28:57 pm
UUID: 42292a5b-3735-fc4b-883b-42249a88aac9
Ancestors: Compiler-nice.93

Ensure usage of #typeTableAt:  rather than typeTable ivar to handle Wide characters

=============== Diff against Compiler-nice.93 ===============

Item was changed:
  ----- Method: Scanner>>scanAllTokenPositionsInto: (in category 'expression types') -----
  scanAllTokenPositionsInto: aBlock
  "Evaluate aBlock with the start and end positions of all separate non-white-space tokens, including comments."
 
  | lastMark |
  lastMark := 1.
  [currentComment notNil ifTrue:
  [currentComment do:
  [:cmnt| | idx |
  idx := source originalContents indexOfSubCollection: cmnt startingAt: lastMark.
  (idx > 0 and: [idx < mark]) ifTrue:
  [aBlock value: idx - 1 value: (lastMark := idx + cmnt size)]].
  currentComment := nil].
  mark notNil ifTrue:
  [(token == #-
+  and: [(self typeTableAt: hereChar) = #xDigit]) ifTrue:
-  and: [(typeTable at: hereChar charCode) = #xDigit]) ifTrue:
  [| savedMark |
  savedMark := mark.
  self scanToken.
  token := token negated.
  mark := savedMark].
  "Compensate for the fact that the parser uses two character lookahead.  Normally we must
   remove the extra two chaacters.  But this mustn't happen for the last token at the end of stream."
  aBlock
  value: mark
  value: (source atEnd
  ifTrue: [tokenType := #doIt. "to cause an immediate ^self" source position]
  ifFalse: [source position - 2])].
  (tokenType = #rightParenthesis
   or: [tokenType == #doIt]) ifTrue:
  [^self].
  tokenType = #leftParenthesis
  ifTrue:
  [self scanToken; scanAllTokenPositionsInto: aBlock]
  ifFalse:
  [(tokenType = #word or: [tokenType = #keyword or: [tokenType = #colon]])
  ifTrue:
  [self scanLitWord.
  token = #true ifTrue: [token := true].
  token = #false ifTrue: [token := false].
  token = #nil ifTrue: [token := nil]]
  ifFalse:
  [(token == #-
+  and: [(self typeTableAt: hereChar) = #xDigit])
-  and: [(typeTable at: hereChar charCode) = #xDigit])
  ifTrue:
  [self scanToken.
  token := token negated]]].
  self scanToken.
  true] whileTrue!